Software Engineering: The Career or Shortage of the Future?

The defining feature of this age in history is the rapid innovation and technological advancements made in the past century. With the invention of the computer, the human experience was revolutionized, and from the emergence of the computer came the necessity for computer programmers and then computer scientists. These engineers have always been in high demand, as computing was popularly thought as the career of the future, but unfortunately, engineers equipped with the industry’s demanded skills have been in short supply. As the 2018 Gartner Incorporated Emerging Risks Survey reports in 4Q18, senior executives ranked “talent shortage” as their primary concern (Lavelle); and this demand is twofold as well, as tech companies demand these skilled engineers to keep up with exponential technological development, and non-tech companies demand these engineers to ensure they digitize as the rest of the world seems to be doing. Chief Executive Officer of tech recruiting company Hired, Mehul Patel even states that “every company is a tech company now” (qtd. by Liu). While this demand would seem to be a great situation for engineers, the result is the opposite as these engineers do not have the skills necessary to meet the demand. In fact, as the demand increases exponentially, and the skill gap remains, there is a growing rift between the increasing quantity of engineers demanded and the slowly growing number of qualified engineers available. My inquiry investigates the reasons for the growing shortage of engineers and the existing proposed solutions. I dive into the benefits and issues with each solution. This investigation into the shortage of engineers can address issues within the technology industry hiring process and issues within the production of engineers, which will give light to solutions that will increase productivity and efficiency as well as promote intellectual and cultural diversity in the field.

The Increasing Demand for Skilled Engineers

Finding qualified engineers is proving itself increasingly difficult as the cutting edge is developing at groundbreaking rates. The 2020 Google Cloud Whitepaper reports that in 2019, 70% of leaders in technology found hiring as a “top challenge” (“Google Cloud Certification…”). Furthermore, the US Bureau of Labor Statistics projects 411,400 new job openings for 2021-2031 employment growth, meaning employers will be facing a more competitive job market as more advanced positions open requiring skilled candidates (“Employment Projections”). However, not only is finding and hiring these qualified candidates difficult, but companies are also struggling to keep their current employees up to date with the skills the developing market demands. For example, the 2021 Global Knowledge IT Skills and Salary Report, which partnered with major technology companies such as Amazon, Google, Microsoft, and IBM, highlights that 76% of tech leaders found large skills gaps among their existing employees (Skillsoft). In short, while it is great for the tech industry to be advancing at such a rate, this development is creating a greater issue if these companies cannot find the required skill to keep up.

As a result of this high demand, a new trend is emerging where employers have resorted to posting about job openings on social media such as Twitter, as Netflix Senior Software Engineer Felipe Barbosa tweeted this September, “My team is hiring! And this position is aimed at less experienced engineers with strong fundamentals and a great attitude!” (Ribeiro Barbosa). Similarly, also this past September, MIT Research Scientist and popular podcast host Lex Fridman posted an announcement on his personal Twitter account, advertising Machine Learning Engineer and Programmer positions for which he is hiring (Fridman). Given that Netflix is ranked 115^th on the Fortune 500, and that Fridman’s podcast currently holds the top spot on Apple Podcasts’ Technology Top Charts, surely such corporations should have no shortage of interested candidates lining up for the opportunity of working with them (Staff; “Lex Fridman Podcast.”). However, as evidenced by both Barbosa and Fridman’s advertising, finding the right engineers is still so difficult, that they will resort to a social media post to find these hires.

The Low Supply of Qualified Engineers

While the demand for engineers increases, the growing shortage can be attributed to a lack of engineers who are qualified and experienced enough to fill the rapidly developing positions. A proposed solution to this is to shift focus from applicants’ credentials and resume fillers to their demonstrated thinking, and this shift can look like increasing the number of technologists and certified applicants hired. The Conference Board non-profit think tank reports finding that employers are “lowering educational requirements and offering more initial job training” in response to the shortage, and in 2013, former UK Prime Minister, and then-London Mayor, Boris Johnson, proposed a “London Visa” that would enable international technologists to work in London and fill the skill gap (“How Employers Combat Labor Shortages”; Flinders). Historically, both technologists and certified applicants are rarely considered for engineering positions, however, many have found that this flaw in the engineering workforce is leaving companies with further unfilled jobs and a loss of perfectly capable and trainable employees.

Increasing Technologists and Certified Applicants Hired

The qualification of “technologist” is often compared negatively to the qualification of “engineer,” with an observable difference in the number of years of education. An engineer is, on an international average, required to take a greater number of years in a training program than a technologist is required to take (Carroll). Similarly, the courses the engineer can expect to take, such as advanced calculus, differential equations, engineering concept creation, and critical thinking, are more advanced than the classes a technologist will take, such as algebra, prealgebra, and engineering principles (“Engineering Technologist vs. Engineer”). However, the most important difference between the two qualifications is the specialization of the graduates. An engineer focuses heavily on the conception of fundamental principles and transforming them into a design, while the technologist understands that design and uses technical skills to implement it (“Engineering Technologist vs. Engineer”). A third category of “technician” exists as a possible qualification as well, however, this position is more technical and less conceptual than the technologist. In other words, the progression from engineer, to technologist, to technician is a spectrum of specialization ranging from the most theoretical to the most hands on. And while each of these positions hold valuable places in the tech workforce, it is the technologist who provides the perfect balance between the conceptual and technical side of the engineering process.

Due to these multidisciplinary skills, a technologist can be a great hire for companies. In the podcast Y Combinator episode “Hiring Engineers with Ammon Bartram,” Bartram, cofounder of the hiring program Triplebyte, explains that large technology companies are able to search for employees who are flexible enough to be trained in the specialization of the desired role (Cannon). This idea follows the logic that, while the engineer may have much conceptual knowledge, they might lack the technical knowledge the company needs, because technical demands are advancing at a rate so that an engineer’s technical experience becomes outdated. Therefore, technologists provide the perfect balance for companies to mold into whatever employee they desire. However, technologists’ qualifications are undervalued because of the systemic stigma that the engineer is superior (Carroll). In his paper, Carroll suggests a reorientation of how technology companies perceive technologists and their qualifications, since a flexible employee with problem solving and critical thinking skills is very valuable to a company.

Another proposed solution is that companies begin to recognize and accept applicants with specialized certifications or professional registrations. Several large companies have begun offering free and quick certifications in response to the increasing shortage of engineers, such as Google, Amazon, Facebook, and IBM among others, and possible fields include Artificial Intelligence, Cloud Computing, Cyber Security, and other fields of the same nature (Leighton; “Microsoft Certifications”). In the previously mentioned interview, Bartram also comments that recently trained engineers, like certified engineers, are likely to be a great match with start-up companies, as these companies’ priorities are to create and implement their project efficiently and rapidly (Cannon). Hiring these applicants is beneficial because since certifications “offer more specific training in a shorter time frame,” there is an elastic supply of certifications and registrations, meaning they can be produced quickly, and graduates are very technical and specialized (Amsler).

The major caveat of increasing hiring of technologists and certified applicants is the risk of overspecialization, which worsens the gap when these employees no longer meet the skill demands. To counter this effect, companies must ensure that training and specialization continue as ongoing process, because if the industry is going to continue developing at this rate, employers must guarantee that their employees are “staying current with trends and technology” (Global Knowledge).

Lack of Representation in Engineering

Another contributing factor to the lack of available engineers is the underrepresentation of minorities in the tech industry. While women of color compose about 40% of the female population in the United States, only 12% of women in the IT sector are of color (“Women and Girls of Color in Computing”). Additionally, a National Science Foundation report found that of all science and engineering workers in 2015, only 28.35% were women, 33% were of color, and 10.4% women of color (National Science Foundation). The issue with this is quite straight forward: there is a portion of the population whose engineering careers and talents are not being maximized; in other words, there is a large number of potential engineers that are not being trained nor hired. Simply looking to these currently marginalized groups when hiring will directly increase the number of engineers in the field because, as NIH Director Francis Collins states, STEM is currently “missing critical contributors to our talent pool” (Collins).

Getting past the historic bias against hiring women and people of color in industries like engineering, there are more concrete reasons as to why minority numbers are so low in this workforce, such as a lack of access to opportunities and a lack of interest and motivation in STEM. In the 2021 Conference on Research in Equitable and Sustained Participation in Engineering, Computing, and Technology, a study on the role of cultural wealth in hiring explains how, prior to the application process, many minority groups do not have access to the same kind of preparations that others receive. For example, the study recalls that many “computer science professors at Historically Black Institutions [(HBIs)] may lack experience with technical interviews” (Lunn and Ross). This creates a rift in how students from HBIs can compete with other applicants, since minorities are unaware and unprepared for these types of interviews, although they are one of the most crucial elements of modern tech hiring (Bui).

Another display of the lack of adequate preparation for minorities can be seen in the disparity of students taking computer science courses in high school and university; only 23% of all students taking AP Computer Science in 2017 were female (20% were of color), and less than 10% of all computing bachelor’s degrees are held by women of color (“Women and Girls of Color in Computing”). These statistics are important because if applicants wish to meet the skill demanded by hiring companies, preparations must take place starting very young, and a lack of access to these resources, according to the Lunn and Ross’ study in hiring, “unequal divides between those with a greater availability to prepare” (Lunn and Ross).

In addition to a lack of access to preparatory opportunities, minorities are also less likely to fill these engineering roles because of a wrongfully influenced lack of interest and motivation to pursue engineering. The Society of Women Engineers (SWE) report on Engineering Messaging to Tween Girls shows that women are “inclined to hold humanistic values,” indicating that if they are not able to see the effects of engineering on society, they are less likely to pursue these roles (Society of Women Engineers). This idea is corroborated by a study seeking to investigate the idea that men prefer working with things and women with people; the study found that the idea in question does hold true, signifying that these preferences play a large influence in “gendered occupational choices and gender disparity in the STEM fields” (Su). Another reason women and minorities are less likely to show engineering interest is because the field is heavily saturated by men, specifically white men, causing minorities to perceive engineering as a “[non-inclusive] profession” (Society of Women Engineers). If the previous reports are accurate, and minorities are perceiving engineering to be a non-inclusive profession that also holds little societal impact, it makes sense that many of these potential engineers are straying away from the industry.

The loss of prospective engineers due to lack of representation is hurting the industry since the diversity and inclusion benefits the industry for a couple of reasons. Logically, increasing hiring from these applicant pools will directly increase the number of possible candidates, which in turn will increase the number of hires; however, more important is the effect this inclusion can have in the engineering workplace. Engineering Associate Professor at the University of Notre Dame Michael Kitz, who has decades of industry experience in companies such as Motorola, Honeywell, and Proctor & Gamble explains that increasing engineering workplace diversity “begins to open up better solutions and better business results” (Kitz). By bringing in diverse hires, technology companies are expanding their horizons to include new perspectives and ideas. This idea is also found in a study published by the Harvard Business Review which reports that “going from having no women in corporate leadership… to a 30 percent female share is associated with a… 15 percent increase in profitability for a typical firm”, indicating that cultural and intellectual diversity is the most beneficial aspect of representation in the engineering field (Noland).

Increasing Representation

There are several approaches in which we can begin to tackle the lack of minority representation in engineering, from early education all the way to the companies who are hiring. In early education, the SWE report found that exposing young girls to engineering in a positive light increases their interest as these efforts “combat [the] negative stereotypes” that deter young girls in the first place; furthermore, this messaging should be diverse and inclusive in its representation of minority racial and ethnic groups to further combat the idea that this field is unwelcoming to these groups (Society of Women Engineers). Such exposure can begin with educators seeking grants so they can fund more classroom technology and integrate more STEM into the curriculum (“Stem Education Grants.”). In addition, incorporating inclusive STEM media and conversations that encourage minorities to pursue STEM are great ways to mitigate the commonly held negative and discouraging perceptions (Society of Women Engineers). Early educators can also join organizations and programs such as Girls who Code, an organization seeking to close the tech gender gap by providing coding clubs for girls across the nation, and contests like Digital Divas, which encourages diversity in young engineers by providing a statewide coding competition for Texas high school girls (“About Us”; Digital Divas 2023).

On the university level, there are numerous scholarship programs available to minorities interested in pursuing a degree in STEM as listed on the North Carolina State University “Funding Opportunities for International, Underrepresented Minority and Disabled Students in STEM” webpage (“Funding Opportunities…”). Lunn and Ross also suggest that educators in universities ensure their students are provided with “increased opportunities for hands on examples and problem solving” to close the preparation gap between minorities and those who are more privileged (Lunn and Ross).

Finally at the corporate level, Luna and Ross suggest companies increase offers of mentorships and internship opportunities to minorities to close their experience gap compared to other applicants. Likewise, Notre Dame Associate Professor Kitz also recalls how companies have made efforts to recruit from minority serving institutions and should increase these efforts to improve their minority representation (Kitz).

Conclusion

While there is a concerning increase in shortage of skilled engineers available to match the growing demand and available jobs, there are several ways to tackle the issue from the supply to the demand end. Issues in finding qualified applicants can be solved by hiring more technologists, who provide flexibility in their ability to be trained and applied as needed, and certified applicants who are very skilled technically and easy to produce. Furthermore, the short supply of skilled engineers caused by the lack of representation in the field can be solved by improving messaging to young girls and POC, increasing awareness of scholarship organizations and inclusive practices in universities, and encouraging companies to play active roles in hiring minority applicants.

Author

Celeste Mannel

Celeste Mannel is from Dallas, Texas studying Computer Science with a hopeful minor in Engineering Corporate Practice. Celeste plans to pursue a career in the tech industry where she can collaborate with teams and develop at the forefront of technology. Inspired by her Argentinian mother who works in the tech industry, and as a Latina woman in computer science herself, Celeste was led by her curiosity in women and minorities in STEM and how they can solve the engineering shortage facing the industry. This essay “Software Engineering: The Career or Shortage of the Future?” addresses the increasing shortage of engineers available to satisfy the growing industry demand, and it argues for solutions that will increase productivity and efficiency as well as promote intellectual and cultural diversity in the field. Celeste would like to thank Professor Whitney James for her feedback and encouragement, Professor Michael Kitz for participating in an interview and sharing his perspective from within the industry, and her mother Hilda Sanz Mannel for her inspiration and constant support.