I scraped 2310 job posts for IT and technology jobs.
Here’s what I learned:
1. Demand For Frontend Developers Is High.
There are more jobs for Frontend developers than Backend or Fullstack developers. Fullstack developers had the lowest count. It seems that employers prefer someone who is a specialist (frontend or backend), rather than a jack of all trades.
2. Javascript And SQL Are In High Demand
SQL and Javascript are the most in-demand languages. This is followed by C++, Java, Python and PHP.
It is interesting to see the level of continuing demand for PHP, which is roughly equal with Python. There is ongoing discussion about the death of PHP, however strong demand in the marketplace still exists. The huge demand for Javascript reflects the popularity of Javascript based frameworks such as Node and React.
There are also clear, niche markets for SAS, Ruby and Kotlin programming languages.
3. React Is The Most Popular Javascript Framework
Vue and React are both Javascript frontend frameworks. As we can see, React is in much more demand. Looking at our graph above for frontend vs. backend developers. It makes sense to learn React if you were to learn frontend development.
4. Node Is The Most Popular Backend Framework
Surprisingly, the demand for Laravel is still strong, roughly equal with Ruby on Rails. If we were to go online, we’d see a lot of people talking about the demise of PHP (and it’s frameworks by extension). But if we look at this graph, we can see that PHP and Laravel are still good options
Overall, Node, which is a Javascript framework has the most demand by a long shot.
5. Communication and Problem Solving Skills The Most Important Soft Skills
Communication is the most in-demand skill by employers. Problem solving is the next most important skill. If you want to get hired you should focus on soft skills, especially communication and problem solving, while a positive attitude goes a long way as well.
The bottom line is people skills are important if you want to succeed.
6. Agile and Devops Are In High Demand
Agile and Devops are the most popular methodologies by a long shot. Some employers prefer the other methodologies. The reason why needs further investigation.
7. Many Jobs Need Microsoft Server Knowledge
Here are three Microsoft server products in high demand.
8. Microsoft’s Azure Is In High Demand
Interestingly, Azure beats out AWS when it comes to cloud services. This is unlikely to be the case in markets overseas where AWS is very popular. A common theme of here is the dominance of Microsoft products in the corporate environment. Bottom line is, get good at Microsoft products if you want a job.
9. Cisco Qualifications Get Jobs
Cisco qualifications are in demand. Specifically the CCDP qualification.
10. There Are Many Jobs For Android Developers
There is a greater demand for Android developers than Ios. This makes sense as the Android market is a lot bigger than the Ios market.
11. Shopify Beats WordPress
Right now, there is a greater demand for Shopify developers than WordPress developers. This could be a distortion due to the Covid Pandemic (as of November 2020) where a lot of brick and mortar businesses are going online, and would use Shopify to do so.
Conclusion:
If you want a job in New Zealand you should consider front-end development, learn Javascript and React frameworks.
There are more jobs for those who specialise in either front-end or back-end development, rather than fullstack.
Node is growing as a back end framework.
PHP is still in demand, despite people saying it’s “dying”. It’s demand on the job market is about equal with Python.
Node, which is a Javascript based backend framework is the most popular. Knowledge of Azure and Microsoft Server is helpful for many jobs.
Communication skills are the most useful skills followed by problem solving skills.
Holding a microsoft server qualification helps. Holding a Cisco certification also helps you get a job.
When it comes to languages, SQL has very high demand, followed by Javascript, C++ and Java.
About:
I’m interested in
- uncovering new markets and hidden demand using data
- finding new trends which can help you save or make money
- making models to improve your business efficiency
Many businesses have huge amounts of data. I can help you turn it into an asset in ways you have not considered.
Let’s Connect!
Methodology (the boring stuff)
Collecting The Data:
The best place to understand the current job market are job boards.
This is where we can see real life demand for skills.
First, we need to collect the data.
I created a simple script using python to scrape job boards.
As I was most interested in tech, I filtered searches to focus on “information technology”.
Tech seems to evolve the most quickly.
The script visits a job search page, clicks on each job post, pulls the description, then appends the post to a text file.
Processing The Data:
The raw text was cleaned using regex to ensure each word was lower case and there were no special characters.
Next we used NLTK libraries to tokenize the text, remove stop words, and produce n-grams.
Getting Word/Phrase Context:
Creating an n-gram is important to work out the meaning a word holds in a sentence.
For example, the noun “go” or “golang” is a programming language, but it can also be a verb.
Looking through the text, when the word “go” appears alongside words like “programming” or “python” or “php” – it’s likely they’re referring to the programming language not the verb.
When “go” appears alongside other words, it’s less likely to refer to the programming language. As you can see, this requires a bit of knowledge of the context of these words.
However, this strategy obviously has it’s faults.
Many of the words for skills in tech jobs do not have multiple uses. For example, “ssis”, “sql”, “scala”, “java”, “kotlin”. These words have only one meaning.
Another problem we faced was some skills don’t exist as single words, but two or more words. For example, “communication skills” and “attention to detail”.
This is where the n-gram was useful – to work out words that appear together.
While doing this, I broke the skills down into categories:
– Programming languages
– Methodologies
– Frameworks
– Frontend vs backend vs fullstack
– Analytics software
– Hard skills
– Soft skills
– Tensorflow vs Pytorch
Getting The Counts:
We’re relying on counts of words and phrases in the text to indicate the demand for a certain skill.
I.e – A high count corresponds to a highly demanded skill.
To get counts we used Pythons Counter library.