Are software engineers really in their last five years of work?

Will we even see doctors replaced by AI one day?

—

Headlines like these suggest that there’s a storm brewing on the horizon even more severe than the potential fallout of a trade war. Could technological job losses accelerate as early as 2027, when some are forecasting the arrival of AGI?

Predictions about the job market are all over the map but often pessimistic. If we are sailing into the uncharted waters of an economy set to be increasingly impacted by AI, some of the scarier forecasts remind me of the illustrations of sea monsters on old maps.

While we may not know the specifics of AI capabilities in the future, we know a lot more know than we did a few years ago. And unlike the sailors and economists of long ago, we also have over 150 years of Census data about how jobs have evolved amidst many previous episodes of transformative technology and geopolitical uncertainty.

News about tariffs and trade seems to be evolving even more quickly than AI these days, and yet economists have been able to quickly build models to estimate the fallout associated with a range of different scenarios1. By taking a more serious look both at the current state of technology, and how quickly occupations have evolved in the past, I think we can also have more nuanced conversations about the potential impacts of rapidly evolving technology— avoiding both exceedingly cautious and distractingly alarming predictions.

I’ve written previously about this fun fact: going back to 1870, the 2010s were the decade that actually saw the smallest shift in the types of jobs that people work (AKA occupational churn). Starting in 2018, these changes picked up again, but in a positive way: they were more driven by faster-than-expected growth in new and growing jobs like data scientists and warehouse workers than job losses elsewhere. The weighted skewness of each year’s shift in occupations suggested that people were more likely to be pulled into new jobs rather than pushed out of old ones. The latest figures from the BLS suggest that this trend ended in 2024 and now more workers appear to have been “pushed” out of their old jobs.

If measuring these technical characeristics of the job market is like predicting the global impacts of climate change, coming up with more accurate forecasts of what jobs are most at risk is like improving our forecasts about the intensity and path of individual storms; a service that could save livelihoods like how good weather forecasts can save lives.

What if we forecasted job losses like we do hurricanes?

A scale like this, based on percent job losses per year or per decade, could help us build a shared language for describing what we might experience:

Grounded in real rates of historic change, this scale is more realistic than other systems. On one hand, it’s more aggressive than official government employment projections, which are only forecasting a single occupation (word processors and typists) as having potential for severe loss over the next 10 years. On the other hand, it is much more reasonable than other private predictions like those on willrobotstakemyjob.com. If those predictions are to be believed, there is over a 50% chance of today’s average job being completely automated within the next 20 years.

If this happened, that would mean a rate of extreme job loss impacting 50 times as many workers as anything that has happened historically. To put that in perspective, only 0.63% of all workers in 2004 were in occupations that even came close to being fully automated2 by 2024.

The reality is that jobs tend to evolve rather than get eliminated, and most job loss occurs at a more moderate pace. But even these moderate losses have had big implications for the job market and society. To give a better sense of scale, I’ll walk through a few examples in each category of intensity:

Mild job loss

In any given time period, over one third of all jobs typically experience some mild decline relative to the average. In a growing workforce, this category would include some jobs that are still seeing some nominal growth that is just slower than the working population as a whole. Where there are real declines, they are slower than the rate of retirement, so plenty of opportunities still open up for new workers. The decline may be nearly imperceptible in higher-turnover occupations that fall into this category like janitors, building cleaners, and grounds maintenance workers.

Like a tropical depression forming over the mid-Atlantic, occupations that were formerly fast-growing but later move into this category are important to watch. Telecommunications equipment operators, after being the hot job of the 1880s-1910s, cooled off between the 1920s and 40s and passed through this category in the 1950s and 60s before entering a more serious phase of decline in the 70s and beyond. 2024 BLS figures suggest that employment prospects for software engineers have already skipped this phase and are now headed into moderate job loss, bringing IT occupations overall into mild decline.

Moderate job loss

Crossing the 2% threshold takes us beyond the natural rate of retirement, but still below typical attrition. Job opportunities may still open up, particularly if the pay is lower and turnover is higher, but most people will know that the opportunities in these fields are increasingly limited. This is quite apparent when looking at the declining number of job postings for software engineers. The rise of online shopping and self-checkout machines had a similar impact on retail salespeople and cashiers in the 2010s.

Bank tellers began their decline into this category a few years earlier, but later crossed into the next category of Severe. While both occupations recovered in 2024, they are historically vulnerable to recessions. Announced job cuts for retail have already been notable in 2025, so the impact of tariffs and deteriorating consumer sentiment makes the whole retail industry an important one to watch. Over the long run, will people’s enjoyment of in-person shopping experiences save retail sales workers from what bank tellers experienced?

Severe job loss

Surprisingly few jobs fall into this category— historically, only a little more than 5%3 of workers have been in occupations that have sustained this level of loss over a decade. However, this category today includes many clerical workers like secretaries and administrative assistants. Declines in the business support services industry suggest that U.S.-based customer service call center workers are also a big part of this group.

The steady downward trajectory of this industry might foreshadow other jobs that involve varied responsibilities, but are vulnerable to both AI and outsourcing. This includes software engineers; or at least those engineers who don’t find their way into management. In the physical world, self-driving vehicles could push taxis and rideshare drivers into this situation nationally by the 2030s, and even earlier in some locations.

Sustained, severe job loss of this order is historically memorable. Many manufacturing focused-occupations fell into this category in the 1980s, the memory of which still haunts our politics today. The high rate of occupational churn between 1910 and 1970 was significantly driven by comparably severe declines in agricultural work.

Extreme job loss

Extreme job loss, which I would define as declines of more than 8% per year relative to overall employment are rare. Sustained extreme declines of more than 57% over 10 years are even rarer, historically happening less than a quarter of a percent of the time. Still, there are some notorious examples, including elevator operators in the 1960s and 70s, and bartenders in the 1910s and 20s. Telemarketers and file clerks have been casualties of the information era since the 2000s and are still in a state of extreme decline. And the impact of generative AI, among other factors has been cited in the recent extreme layoffs of special effects artists and animators which sent the occupation into an eye-watering 30% decline in 2024.

Single year extreme declines are more common that decade-long ones though. Totaling up all of the episodes of extreme job loss each year gives us a good snapshot of some of the biggest drivers of unemployment at each point in time.

A big temporary drop in construction work is very visible between 2008 and 2010, alongside the more permanent decline in production workers like machine tool operators. Slow, continuous losses among various clerical occupations due to technological change are also noticeable. Many of these, like office machine and telephone operators have suffered extreme losses in some years and merely severe losses in others.

Since these occupational numbers only come out once per year, the trends will first become apparent in monthly industry numbers4. And there are even more forward looking indicators, including layoff announcements, big drops in job postings and even bi-weekly employer headcount plans, all of which can help us anticipate emerging extreme job losses before they officially happen.

The big picture

If there’s something that can be learned from all of these historic examples of extreme job loss, it’s that jobs that are highly dependent on the continued existence of a particular technology are the most vulnerable. Phones with no caller ID were a boon to telemarketers, and we needed more photographic process workers before the instant gratification of sharing photos on Instagram existed. It’s easy to see how AI even in its current form could further accelerate losses in more specialized, repetitive, information-focused occupations while taking a longer time to have an impact on more generalist occupations that bridge information with the physical world in less predictable ways.

But government is another significant driver of change. The absolute nature of laws can push changes that tend towards the extreme end for better or for worse. Historically, Prohibition cratered the job prospects of bartenders (while creating new illegal opportunities). More recently, announced and anticipated DOGE cuts and layoffs are set to push the civilian federal government into “Extreme” job loss for the first time since World War II demobilization.

While markets and AI-assisted Youtubers reacted strongly to the uncertainty around tariffs, the wide-ranging impacts of federal government cuts may not yet be fully appreciated. Nearly half of all federal civilian workers are in highly specialized occupations the are disproportionately (20%+) government, from air traffic controllers and postal service workers to conservation technicians and budget analysts. Individual circumstances will vary, but the different skill profile of federal employees, contractors, and grantees may take some time to be absorbed into the private sector. This means that we’ll likely see an uptick in either unemployment, occupational churn5 , or both over the coming year, along with the many other consequences of these drastic policies.

Would these cuts on their own be enough to spark a recession? How many manufacturing jobs can realistically be reshored to the US? How do the timing of these events interact with other factors like interest rates and AI advancements that have already pushed jobs like animators into extreme losses?

Using the framework here, we will continue to explore these questions at 2120 Insights over the months to come.

But how many people typically lose their jobs each month, and how will we know how much of these losses are technology-related compared to other factors?

It might be surprising to learn that the total rate of job turnover is nearly 40% per year1 and sometimes higher!

Typically most of this turnover is voluntary— layoffs exceed quits only in the deepest parts of recessions. And during these times, the total amount of turnover actually decreases.

Intuitively, this makes sense. When layoffs are happening everywhere else, who wants to quit their job?

This pattern of people becoming more reluctant to quit can sometimes precede a recession as it did in 2007. Job openings and hires were the first to fall, followed by quits, and then ultimately layoffs. As we can see in the chart above, a similar pattern has happened more recently as the vibe shifted from record-high quits during “The Great Resignation” in 2021-22 to recession-era levels of hiring during “The Great Stay” starting in 2024.

It’s not in the above chart, but more advance indicators like the Challenger job cuts report show that layoffs are starting to pick up too, mostly in government and retail. This will be a leading indicator of the official JOLTS figures to come2.

It’s true that layoffs before February 2025 were still historically low. The baseline was much higher even in the otherwise strong economy of the mid-2000s as there were significant job cuts in manufacturing and some types of office/administrative work. But just returning to that baseline from 2024 levels would be a jump similar in size to the one that happened between the mid-2000s and the Great Recession. If the hiring rate was the same as it was in 2006, returning to 2006 levels of layoffs would be less concerning. But low hiring is just what one might expect before a wave of automation— companies will just stop hiring before they lay anyone off. The same is true if there is considerable uncertainty about interest rates and tariffs in any export-oriented or import-sensitive industries— companies may prefer to wait out the uncertainty rather than make big immediate investments. And indeed, the Fed highlighted this uncertainty in its March 2025 call.

For a deeper dive into high-frequency labor market indicators and charts like these ones, I highly recommend

The details of how much delayed investment vs. technological change might hold back hiring and accelerate layoffs is something that requires more careful examination. But at a high level, both of these combined with changes in consumer preferences are important factors in determining the rate of occupational churn3 over the long run that we might expect to see in “robots taking jobs” scenarios.

As I wrote last spring, the 2010s saw record low levels of occupational churn, but this is on the the rise again— an observation also confirmed by , Christopher Ong, and Larry Summers in their paper “Technological Disruption in the U.S. Labor Market”.

But how much is this “churn” compared to the overall rate of layoffs? And what does it mean if technology or other factors push it up?

If we look at the total amount of change across individual comparable occupations, we see that it’s much lower than the 11 to 21% per year overall layoff rate.

Occupational churn is usually somewhere between 2 and 4% in individual years, and under 2% annually when measuring true long-term shifts in occupations over a 10-year time frame.

Notably, changes in occupation were much higher during the pandemic, especially relative to the low levels of layoffs in 2021-2022. But most people return to the same occupation because most layoffs are seasonal. Construction and hospitality work slowing down during winter is a particularly visible example of the significant difference in layoff rates across industries.

A secular acceleration in layoffs, whether they are due to tariff-sensitive export-oriented industries, cuts in government staffing and funding, labor-replacing technology, or low consumer confidence really could upset this precarious balance. While it’s true that over the long run, workers in dying occupations who get laid off tend to find a job elsewhere, the new jobs don’t always appear in time, especially when hiring is low as it is now. The most forward-looking indicators of future hiring like business headcount plans and job postings on Indeed have been trending in an unfavorable direction.

If the current generation of AI ends up doubling the long-term rate of occupational churn (bringing us back to the rapid changes seen between 1910 and 1960), that alone would displace about a third as many workers as the Great Recession did4. When combined with second-order effects and other factors, it’s easy to see how much of an impression such a shift would have on society.

Of course, proponents of an early arrival of Artificial General Intelligence (AGI) are making even bigger predictions. If the uncertainty of today’s “Great Stay” is like when the sea pulls back before a storm, how can historical data help us make a better forecast of what might actually happen when the storm arrives? How can we come up with a better understanding of the magnitude of the possible changes ahead? How might the timing of other important geopolitical factors affect this? And what does rebuilding look like after the storm passes (or continues)?

I’ll be proposing a few frameworks over the coming weeks that I hope will guide these sorts of conversations, and bridge the gap between those concerned about AI’s potential impact on jobs vs. those with more conventional concerns about policy and the business cycle.

Estimates of Waymo’s market share vary widely, but I suspect that even after a year of impressive growth in a tech-friendly city, it still only represents about 10% of total San Francisco rideshare volume1. Monthly growth has been between 13,000 and 66,000, meaning it could take anywhere between 0.25% and 1.5% market share each month, or between 3% and 18% per year.

Taking 18% of each local market’s share each year would put Waymo on a similar trajectory as rideshare was nationally between 2013 and 2018, when the Census recorded a doubling of the number of taxi drivers and chauffeurs. But even this falls short of the tripling that was observed both in the San Francisco metro area and statewide in California.

On the other hand, only gaining an additional 3% market share per year would mean that the recent slowdown in growth could be reflective of a limited appetite for self-driving taxis, which might happen for a number of different reasons. Let’s take a closer look at these risks, both near-term and long-term for both drivers and the self-driving car industry.

In the near-term: safety first!

There has been good evidence that Waymos have been safer than an average human driver since 2023, when first did some in-depth reporting on this topic. Not long afterwards, a study by the insurance company Swiss Re in 2023 found that Waymos showed a 76% reduction in property damage claims. A follow-up study published in December 2024 showed further improvement to an 88% reduction in these claims2.

2025 is the make-or-break year for Waymo. Based on their current trajectory, they will likely surpass 100 million autonomous miles this year. If they significantly exceed this without being at fault in a fatal accident3, they will have an increasingly strong argument that their cars are definitively safer than the average human driver in the situations in which they operate.

That said, the mileage that self-driving cars have logged so far has been disproportionately on local roads in warm weather. Because these cars are being introduced to different environments one at a time and might experience different failure modes in each one, each significantly different environment might require us to set that count back to zero. This might be why even though Waymo employees have been able to take their self-driving cars on freeways since January 2024, there is still no timeline for this high-stakes, high-speed driving to be available to the general public in March 2025. Reliably driving in snow and ice is another situation where the clock may need to be reset. But with many more autonomous vehicles on the roads now than there were a few years ago, we will be able to see these milestones realized more quickly.

This scaling is indeed happening more quickly than before— Alphabet CEO Sundar Pichai expects Waymo to be “operating in 10 cities robustly” by the end of 2025, including its first expansion outside the US in Tokyo. Unexpected safety issues could delay Waymo’s timelines, but the limited availability of suitable vehicles and other logistical issues will still set an upper bound on how much more change is likely to happen in the near term.

Medium term: is the price right?

Rumored to cost somewhere between $150,000 to $200,000 each after outfitting, Waymo’s Jaguar I-Pace cars aren’t cheap! Uber CEO Dara Khosrowshahi believes they are currently too expensive to commercialize, so current demand might be more based on the qualitative aspects like having one’s own space, getting to pick the music, and the overall novelty of such a futuristic experience.

I agree with this take, and would add that even if scaling brings their costs down, it will be difficult for them to get much more than 20% cheaper than a human-driven ride. The percentage of fares that drivers take home after expenses is actually pretty low— after taking a deep look into this, I would agree with studies that suggest that the average driver makes roughly minimum wage4.

Zooming out to Uber’s breakdown of its gross bookings gives us a clearer picture of how the expenses of self-driving taxis stack up compared to human-driven ones.

This chart shows that about 68% of fares go to drivers. But even at the cheaper per-mile rates of EVs, 18% of this goes towards electricity, repairs and cleaning, and 34% goes towards paying for the car itself5, both of which Waymo is just as subject to as rideshare drivers. So that only leaves 48%, or 33% of total fares as true wages, plus 6% for tips6.

So even if Waymo’s vehicles cost the same as rideshare drivers and they had zero operating expenses or profits, and their safer driving record lowered their commercial insurance rates by over half (4% of fares overall) that only leaves room for a 43% price cut (though this is closer to 60% for more generous tippers). But of course these expenses are real.

Without more detailed data on Waymo’s operating expenses, I would assume that per-ride even after scaling up, it would be about twice as much as Uber’s. Not only do they have to develop and maintain dispatching software in markets like San Francisco where they operate independently from Uber, they also have to develop and maintain self-driving software and a remote operations team7 , as well as physically cleaning and servicing the vehicles. And they arguably have to work even more closely with local government than Uber does in order to operate successfully. All of these expenses would chop another 18% off any potential savings, at least in the near term.

And this 18% doesn’t even include any additional costs for the specialized vehicles themselves. If even the cheaper-than-average rideshare vehicles make up 23% of the fare cost, the Jaguar I-Paces that are 5x as expensive would already suck up the entire fare cost even if other factors are ignored. A successful self-driving taxi company would need to ensure its cars are no more than double the cost of the average rideshare vehicle if they wanted to compete on price at all, though 3x the price (which could be true of the next generation fleet of Hyundai Ioniq 5s) would still be competitively priced for those who would otherwise tip at least 20%.

Getting to that “last mile” where self-driving cars are actually cheaper will require gradual efforts across multiple fronts. Scaling up might get them to 2x the cost of rideshare vehicles or less, though there is a lower limit8 to this given all of the additional sensors and computing power that a true self-driving car requires. Over the longer run, it’s conceivable that more compact, specialized 1-2 seater vehicles could even be cheaper both to purchase and operate than the sedans typically driven by rideshare drivers, though at the cost of the additional safety concerns that come with smaller vehicles.

Self-driving taxis will first be competitive in cities like San Francisco where cost of living and wages are higher, but wages increases in general will favor more automation over time. A gradual reduction in the 18% of fares that go towards platform operating costs and profits will favor this too. On the other hand, high minimum insurance requirements and special taxes on self-driving taxis would temper the incentives that people may have to switch to self-driving taxis.

All of these incentives are much more gradual9 than the significantly increased convenience (as well as lower prices) that initially drove people onto ridesharing platforms in the 2010s. This is a little paradoxical given that the technology seems much more magical in many ways. But we’ve seen the early returns to this in San Francisco— even after over six months of operations open to the public, market share is hardly above 10%. Lower prices are an important missing factor right now, but any company looking to succeed in this space will need to offer those prices and a trustworthy service.

The X factor: public perception and trust

Even if self-driving cars are statistically safer and cheaper than human-driven ones, there is the question of media coverage and perception. Will our tendency to closely scrutinize something new keep skepticism of self-driving cars high? suggests that it won’t be sufficient for self-driving cars to be statistically safer; each major crash will need to be seen through the lens of “Would a human have done better?”

There is a lot of evidence to suggest that the public isn’t anywhere near as excited about self-driving technology as Silicon Valley is. The annual AAA survey in 2025 showed that 61% of American drivers were still afraid of self-driving cars, and only 13% trusted them.

Though this is an improvement from 2024, there is still a long way to go. Those who take rideshare more regularly might be slightly more trusting, but it’s still worth noting that 50% of Uber users in Phoenix, where Waymo has been open to the public since 2020 choose NOT to take a Waymo when one is offered to them in the app. This will likely come down if self-driving cars consistently demonstrate a safer driving record and lower prices over the coming years, but it’s important to recognize that some people will remain very uncomfortable with the idea of riding a self-driving vehicle.

And even among people who are comfortable with self-driving cars, there will still be a number of situations where having a human driver will be preferred. As long as self-driving cars are constrained (and rightfully so) by stricter rules, there will be many cases where people will prefer the independence of human drivers who make calculated risks. Waymos are designed to always stay under the speed limit and occasionally need to pull over and await human guidance in ambiguous situations. They might be harder to escape in incidents of harassment, or other situations where people take advantage of the cautious programming of a self-driving car. A human driver willing to aggressively merge onto a highway, or speed up to make a yellow light may continue to be the top choice of travelers in a hurry for some time to come.

As the self-driving taxi market grows, there is also the question of how scrupulous the competition will be. Even though Waymo has taken great care to avoid serious accidents, and worked closely with local government to mitigate the risks of edge cases, a newsworthy incident with one of its emerging competitors (including Tesla10) could spook potential riders. In the words of , self-driving car companies win alone, but lose together.

And beyond safety, there are also other macro concerns. If technology ends up throwing large numbers of people out of work, self-driving taxis could be a visible symbol of this and thus vulnerable to more protests like these ones that might discourage ridership. Alternatively, they could fall into the crossfire of partisan politics, speeding adoption along in some areas while slowing it down elsewhere.

The bottom line: how many jobs will robotaxis take?

Assuming that at least 50% of Americans don’t trust self-driving taxis now, and there are a further 10-30% of scenarios where otherwise trusting people might prefer a human driver, we can put a ceiling on automation somewhere between 35% and 45% over the coming years. But until self-driving taxis become noticeably cheaper and can operate in most places (including snowy northern cities), we won’t even come close to this ceiling. Once we do, say in the early 2030s, we might see some “normalization”11 of self-driving technology that will raise this ceiling over time.

This scenario corresponds with roughly 6% job losses per year among drivers. Adoption of self-driving taxis might grow faster than this as the overall demand for mobility services increases, but 6% losses would mean that drivers face a similar situation as bank tellers did when the ATM was introduced. At first, the rollout was slow and bank tellers did find some other work to do, but by the Great Recession we did really start seeing a major shift12. I expect the impact of self-driving taxis on the job market to be similar. We might be seeing this 6% decline already in San Francisco, but it might take until roughly 2030 for the national average decline to hit that level.

I should add that this prediction and the others that I make below are at my 25% confidence level— I think these are all quite possible, but safety incidents or political impacts could delay this change by as many as several years. It’s possible that what I am predicting for 2030 won’t happen until 2035 or later, but it is unlikely to happen much earlier. Adjust your own forecasts accordingly.

In terms of market forecasts, Waymo’s $45 billion valuation in October 2024 was about one third of Uber’s. Similarly, their $11 billion invested at that point was about one third of what Uber lost since 2014 before finally turning a profit in 2023. If self-driving taxis are a similar winner-take-most market, their valuation relative to Uber is about what I expect their relative market share to be around 2030— 25%. To Google, the upside of possibly surpassing Uber as a competitor13 by the mid-2030s (or at least earning a comparable amount as a partner) must be roughly in balance with the possibility of significant delays or loss of market share from competition.

Measuring and anticipating the impacts

Interestingly, these projections bring us back to where taxi driving employment was in the pre-smartphone era. Still, this is potentially a 0.2% increase in unemployment nationally, and closer to 0.5% in certain major cities— a (slower) mirror image of how the rapid rise of Uber may have contributed to the lowest rates of unemployment the US has seen since the 1960s.

And just looking at these unemployment numbers understates the impact. For every full-time driver, there are at least two part-time ones14. And these part-time drivers, who were much less common in the pre-smartphone era15, have a slightly bigger impact on the economy than the full-time ones16. So 50% automation of ridesharing would really impact 1 in 100 workers, or even 1 in 50 if you consider how many part-time (and even full-time) drivers only work for part of the year or between jobs.

This high turnover in rideshare driving (over 60% per year for Uber) means that competition from self-driving cars won’t affect career drivers as dramatically as automation and offshoring impacted factory workers in the 1980s. But a reduced amount of “gig” work will cut off an important source of flexible income for many people that has been shown to improve worker bargaining power in general. Considering how negatively generative AI tools affected the pay of TaskRabbit workers (almost immediately), it’s quite possible that we’ll see the technology of the 2020s take away some of the “gig” opportunities of the 2010s. This includes another occupation that was greatly accelerated by both technology and the pandemic: delivery workers.

Delivery workers in even greater danger?

Self-driving taxis competing with rideshare drivers is just the tip of the iceberg of the impact of autonomous vehicles. The number of delivery workers (or “Couriers and messengers”, as the Census calls them) surpassed the number of taxi drivers and chauffeurs in 2020:

The number of jobs here began falling again in 202317, most likely due to people dining in more rather than ordering delivery post-pandemic. But how soon will delivery robots start cutting into these jobs too?

Starship Robotics, one of the leading companies in this space, passed 7 million deliveries by the end of 2024, ahead of Waymo’s 5 million. And even if their growth appears to be stalling, they have numerous competitors already operating in many cities worldwide, from sunny LA and Dallas to snowy Chicago and Helsinki. Coco’s fleet has notably grown from 100 to 1,000 robots between 2022 and 2024, and the revenue of these companies is estimated to be around 1% of the total online food delivery market.

That said, I think automation in this space will lag behind self-driving taxis, at least at first. If we compare $8,600 per robot to the cost of a car, a delivery robot may indeed bring operational costs down by half. Being small and limited to only 5 mph on sidewalks makes these robots much cheaper than a Jaguar I-Pace, and makes regulatory approvals a lot easier. But their low speed and short range means competition isn’t so much human delivery workers driving cars, it’s human delivery workers on foot or bike! These sorts of deliveries are only feasible in more densely populated areas, so their market size and cost advantage is not as great as it seems at first.

But similar to how rideshare apps paved the way for delivery apps to grow a few years later in the 2010s, I expect that once self-driving taxis are cost-competitive with rideshare, full-sized delivery vehicles will follow closely behind. Companies like Avride are working on both at the same time.

By the early 2030s, I think automation here could proceed even more quickly as specialized economical self-driving vehicle design gets scaled up, service is available in most areas, and most people (even those who might not get in a self-driving car themselves) would accept a delivery from a robot. Possible complications with the logistics of coordinating with customers would be the one thing holding back automation from proceeding at 10% per year by then (more than this is rare18).

With all this in mind, we should more than double our expected impact of self-driving technology from roughly 1 in 500 workers to 1 in 200, noting that many more part-time workers will be impacted too. But this doesn’t include the even bigger impacts that would be seen when robotics and self-driving technologies advance enough to displace workers in logistics and trucking.

The ripple effect on logistics, trucking, and public transportation

Stockers and order fillers, along with pickers were some of the hottest jobs of the 2010s as Amazon warehouses popped up around the country, employing nearly 10 times as many people as taxis and rideshare at the height of the pandemic and growing even alongside an increasing number of robots. But a mix of the post-pandemic retail recovery and increased automation19 caused sustained job losses in the warehousing industry for the first time since the Great Recession.

While these leveled off by late 2023, new robots capable of lifting packages up to 50 pounds were introduced to dozens of warehouses in late 2024, with additional robots likely to be delivered in future 2-year development cycles. Amazon represents over one third of total warehousing jobs, so these are significant developments, even if they won’t be in the public eye to the same extent as self-driving taxis and delivery vehicles. However, it’s important to keep in mind that previous robotic enhancements didn’t displace workers, and many stockers and pickers work outside of warehouses too. I expect automation here to proceed more slowly than with driving and light delivery— declines in employment might just proceed at the rate they have since 2021. But if technological unemployment picks up here, it will likely be forecasted by Indeed postings for Loading and Stocking jobs and confirmed in monthly BLS jobs reports for the warehousing industry20.

Trucking is the biggest of all of the transportation-related occupations— nearly 3% of the total US workforce! As explored in September 2024, there are a number of companies pursuing different strategies in the autonomous trucking space. Aurora’s official launch of driverless trucking service in April 2025 in Texas will be something to watch closely. They have a roadmap for expansion at scale starting in 2027, and the economics appear similar to that of self-driving taxis with driver pay being about 30% of costs, so it’s conceivable that they could only be a few years behind Waymo. On the other hand, being less far along means more uncertainty. There are also many specialized use cases to account for and truckers’ unions might succeed in legally restricting the use of autonomous trucks. So I don’t think this transition will be as quick as it will be for taxis and food delivery. Still, there is potential for job losses of up to 40% by 2040, which would bring unemployment up by 1% in the absence of other work.

I also expect the number of bus drivers to decline, but probably close to the rate of retirement, around 2%. The majority drive school buses, where demand is already decreasing because there are just fewer children, a trend which appears set to continue for the foreseeable future. In a future where more people trust self-driving cars, school bus driving jobs might evolve into something that might end up being classified as that of an attendant, more focused on keeping an eye on students rather than the road. I could imagine a similar future for passenger and public transit bus drivers, where smaller and more frequent autonomous buses and trains could increase frequency and freedom. Instead of driving, operators might have more of a community ambassador role, ensuring smooth operation of their lines without having to be present in every vehicle. In other words, one could envision as does that transit operators would become managers of mobility. This will be especially critical if cheaper mobility means more traffic, an outcome from the initial expansion of rideshare in the 2010s that would be better to avoid.

Taking all of these projections together, it’s possible that we could see a 35% drop in the percent of workers in transportation and moving-related jobs by 2040. Assuming that the other non-managerial jobs in this sector not mentioned here also follow a similar path, that’s about 1 in every 37 full-time workers displaced, with even more impacted on a part-time basis. For the sector as a whole, this would be unprecedented:

But it echoes what happened to rail workers in the mid-20th century. 1 in 77 US workers played some role in operating trains in 1920. This had fallen to its modern level of less than 1 in 1,000 by 1990. We can imagine where a similar trajectory might lead us to by the year 2120.

Moving from rails to roads transformed daily life in the 20th century. Moving from manually driven vehicles to autonomous ones promises a similar transformation in the 21st. Fewer accidents, less need for parking, and improved public transit are particularly exciting prospects. And while a 35% decline in jobs between 2024 and 2040 would impact millions of people, spread out over 15 years it amounts to about 2.6% per year, quite similar to the initial rate of decline in railroad work between 1920 and 1950.

A 2.6% decline isn’t quite as dramatic as what some in the tech industry are forecasting. But there is a difference between what might be theoretically possible and what will happen in reality. My predictions for change are much closer to ’s predictions of AI’s impact on the economy. While this might seem slow to someone who predicts 10% change or more in a year, 2.6% of all transportation and moving jobs is still about 300,000 jobs per year, comparable to cutting 10-15% of federal government jobs21 each year— this is a lot, especially after you consider how many other jobs will also be disrupted.

On the other hand, a 2.6% decline is only a little faster than the rate of retirement, and slower than typical attrition from many occupations. This means it will be less about displacement of existing workers and more of a challenge for younger people entering the workforce, a story that other economists like have been following. It might also mean lower wages for existing workers unless job growth or reduced working hours elsewhere keeps unemployment low. But there will be second and third-order effects on other occupations too, from auto mechanics to restaurant and hospitality workers in truck stop towns. The way these changes will transform livelihoods is something that will be critical to keep a close eye on over the coming years.

12

The decline in bank tellers since around 2010 is more of a return to the 1980-2000 trend.

18

Sustained employment declines in occupations over 10% per year are quite rare. Here are some of the more notorius cases. Only telemarketers in the 2010s declined by more than 10% per year on average.

The value proposition of delivery robots just isn’t quite as strong. Even if expansion gets quite aggressive in the 2030s, I doubt it will automate away any more than 8% of delivery jobs per year on average.

And yet if you are making decisions about what to study, or what career to focus on, thinking about what sort of job you might have in another 10, 20, or even 50 years is unavoidable. In a world with such a hazy future, what jobs might still be around in the 2070s and beyond?

After spending the past few years studying how much jobs have changed in each of the past 15 decades, and keeping a close eye on 6 key AI variables, I think it’s helpful to think of the jobs of the future belonging to at least one of these 3 (or maybe 4) broad categories:

#1 Inherently human jobs

Think of these as all jobs where having a human is inherently valuable to the customer.

Whether this is a matter of trust, rapport, motivation, nostalgia, signaling that one has enough money to hire a person rather than a robot, or even a moral preference, these jobs are already a significant fraction of the economy and may be more resilient than many expect, especially given how adoption of AI might actually prove to be unpopular. These “inherently human jobs” span a range of fields, from therapists/counselors to fine dining restaurant staff to babysitters.

It’s worth noting that the three sectors that contain the highest proportion of these jobs comprise about 30% of overall employment and have been rising over time:

#2 Knowledge verification jobs

This is primarily technical work where personal accountability for factual accuracy is required.

Whether AI hallucinations can ever be fully “solved” is a hot topic of debate among AI researchers. But even if they become exceedingly rare based on benchmarks like SimpleQA, we are still likely to require subject matter experts. Someone will need to verify that AIs are indeed still providing us with accurate information that reflects an updated knowledge of the world, filling in any gaps or vulnerabilities that may prove unexpectedly difficult to seal off in real-world situations, especially in fields like law where an extremely high level of accuracy is often required. There is a trust component to this as well— decision-makers may inherently trust a fully accountable individual more than a loosely accountable machine. It’s been demonstrated that people set a higher bar before trusting a robot, and that trust is more easily broken.

Many technical jobs in law, medicine, science, engineering, technology, and auditing fall in this category of “knowledge verification”. The late 2020s and early 2030s will likely see an ongoing reassessment of the jagged frontier of when an answer from an AI might suffice vs. which problems require an accountable human. What happens to software engineering jobs, where the largest part of the corpus of operational knowledge is already in AI training data, will be an important leading indicator which we can track via a mix of job postings data and industry employment data. While I would be surprised to see this occupation disappear entirely, I wouldn’t be too surprised by either a modest growth (if new tools make knowledge-intensive work cheaper and induce more demand) or a modest decline if exisiting human expertise paired with AI is sufficient.

What is certain is that education and training for these roles will need to be changed significantly to better complement new technology as it evolves— after all, developing subject matter expertise involves a heavy personal investment and it’s important to do it right, especially if more frequent adaptation is required. And if more “knowledge verification jobs” disappear than expected, it may be because these evolve into management roles as the scope of managerial duties is set to increase in an era of powerful AI.

#3 Managerial jobs

This includes any work that requires highly consequential decisions to be made by a directly accountable individual.

Just like the two types of work above, this will take many forms, from jobs required to exist by regulation to entrepreneurs who might employ a mix of people and AI for their ventures. Similar to knowledge verification work, the core principles supporting the continued existence of these jobs include accountability for decision-making and desire for control, though some technologies might reduce the need for line supervisors and middle layers of management over time.

Of course, management jobs are rarely entry-level. If you are starting your career today, your journey will likely start in you specializing in an “inherently human” or “knowledge verification” job, or possibly though another fourth category: “other physical work”.

Other physical jobs (#4?) is an honorable mention here. This includes jobs that require physical work, specific expertise, and need to be performed in unpredictable environments, but don’t fall in any of the other above categories. Some types of work that fall under this such as driving may be significantly automated by self-driving vehicles within the next decade or two. Other types of work such as construction and repair may not be significantly impacted until cost-effective, skilled humanoid robots become available, but that would require developing robots that have a very wide range of physical skills and even after that there are maintenance concerns to consider. It’s possible that more investment may be made in developing complementary robots for some types of construction and repair work.

In the meantime, job prospects in this field are likely to be solid over the next few decades, especially in data center and electrical infrastructure construction as well as semiconductor production. The major caveats are that exact levels of employment will be sensitive to interest rates, and advancements in humanoid robots are somewhat of a question mark over the very long run.

The bottom line

Good analyses of the automation potential of different jobs often include a percentage estimate, which I think is an important way to think about this. Take restaurant workers for instance. Fast food restaurants, where customers care more about a cheap and tasty final product could be largely automated in the not-too-distant future. But sit-down restaurants might only be partially automated, based on the tastes of their customers. This might in theory eliminate half of food service jobs.

But 50% automatable doesn’t necessarily translate into 50% fewer jobs. If the same technology that causes this automation also causes society to become wealthier as a whole (assuming gains are shared reasonably equitably, another big if), the demand for the remaining human-operated sit-down restaurants could go up. If this demand ended up doubling, it would bring us right back to where we started. On the other hand, the rise of economical domestic robots that can do home cooking might cause these restaurants to take a hit. Will such robots be capable of self-repair, or as points out, would more robots mean more jobs open up in servicing these machines?

Thus we return to having to accept a high level of uncertainty about the future of work, at least over the very long run. If there is one thing that makes me optimistic, it’s that we are capable of tracking the early signs of technological employment more closely than ever before. Hopefully a mix of better predicting the impact of new technology on jobs, rethinking re-skilling, and thoughtful policies aimed at keeping inequality in check can help guide us towards a future where people are able to spend more time doing what they love.

But technology creates new jobs too. This isn’t always a 1-to-1 relationship— sometimes jobs are destroyed more quickly than they are created. But how might this play out over the next 10 years?

The Bureau of Labor Statistics (BLS) publishes a list of the fastest growing jobs, but their projections have historically been conservative, particularly in under-predicting the fast growing jobs of the 2010s like data scientists. Here are three jobs that might surprise us over the coming decade:

Biologists and medical scientists

Ever since the success of the Human Genome Project in the 1990s, it has been said that if the 20th century belonged to physics, the 21st belonged to biology. The extraordinary drop in the cost of genome sequencing over the past few decades has facilitated many discoveries from better understanding genetic illness to the use of CRISPR to edit the genes responsible. And given AI’s ability to quickly synthesize information and be employed in new types of experiments (plus impressive complementary advances in augmented reality), increasingly complex problems in the fields of neuroscience as well as biology and health in general could soon be solved.

From the perspective of the job market, the number of life scientists1 also surpassed the number of physical scientists in the 90s, and this trend keeps on continuing.

However, the BLS only projects a little over 30,000 new jobs in this field by 2033. That’s a lot less than the 77,000 that were added in the previous 9 years. While higher interest rates have put a damper on hiring since 2023, job announcements in scientific research are down about 12% relative to pre-pandemic, and the government funding that is responsible for about a third of overall R&D spending in life sciences is subject to change, it would be quite surprising (especially to the increasing number of grad students) to see such a slowdown persist for a whole decade when new technologies appear to be quickly expanding research possibilities.

The forecast

A more reasonable forecast should weigh the political and economic constraints of today against recent Nobel Prize-winning discoveries such as AI-enhanced protein design, and the greatly increased rate of discoveries in materials science. Innovations like these may over time compensate for the more uncertain political and economic environment. It’s also worth noting that life scientists are only a little more than 5% of overall pharmaceutical industry employment, so further efficiencies in other areas such as administrative work and new approaches to clinical trials might result in more resources being allocated to science, and at the very least allow the growth trend from the 2010s to continue. That would mean somewhere between 80,000 and 100,000 more biologists and medical scientists, roughly triple the BLS’ official forecast.

Counselors and therapists

With mental health taking a turn for the worse in the 2020s, along with even more people seeking treatment, one of the top 20 fastest-growing jobs are therapists, or as the BLS more specifically calls them, “Substance abuse, behavioral disorder, and mental health counselors”.

They predict about 19% growth by 2033— somewhat faster than the wider occupation of counselors2 at 12%, and much faster than the 3% growth predicted for the whole U.S. economy.

While this is 124,000 new jobs, such growth would still be a slowdown from the mental health crisis and the explosion of telehealth options that accompanied the pandemic. And yet jobs in the social assistance industry have returned to their longstanding trend of roughly 5% growth per year— almost 3x as fast as the 1.8% annual growth that the BLS predicts. Looking further forward, job postings in the community and social service sector tell a similar story. Google search trends for “mental health”, “therapist”, and “anxiety” remain high.

Estimating how much unmet need still exists explains why there is still so much growth. In 2022, 13% of American adults— nearly 33 million— reported seeing a mental health counselor. But this is still significantly less than the 23.1% who reported experiencing any mental illness that year. Cost and the difficulty of finding a provider are the most significant barriers even as salaries for counselors are low relative to the many years of education typically required. Low Medicaid reimbursement rates in some states, as well as strict degree requirements for Medicare reimbursements are a significant contributor to this problem, as these plans cover roughly 40% of Americans3. However, recent policy changes have expanded the range of counseling that Medicare plans are able to provide.

Future changes in government policies are hard to predict, but the complexity of finding a provider is a wicked problem that better tech still has some potential to solve. The process of finding the right professional for one’s unique circumstances in life can be an arduous one, especially for someone already struggling with mental health issues. And while ChatGPT may be a convenient option that a growing number of people are interested in (and which 80% of those who have tried have found effective), such tools might complement rather than substitute the human connection that is typically needed for recovery. But there are a few other complementary factors that might help expand access to counseling:

• AI could bring down the software development and administrative costs for providers while enhancing the patient experience

• A second wave of mental health tech might build on the lessons learned from the first one, including incorporating more clinician feedback into the wide variety of tools that exist

• Better information sharing across disciplines might result in more targeted, effective, and economical therapy options that open the door to more patients

• Insurance companies, instead of trying to avoid paying providers, might recognize the connection between mental and physical health and prioritize getting more patients the preventative mental healthcare they need so as to avoid more expensive physical ailments

• More people from under-represented-in-counseling backgrounds (in particular men, who are less likely to report seeking care) could enter the field

At least some of these factors are likely to support continued job growth in counseling. And beyond just addressing the current deficiencies, there is a real possibility of quality improvements as well. While 85% of Americans who have sought out counseling have found it at least somewhat effective, there are a number of new technologies and initiatives in development that might further support the mental health professionals of the future. Perhaps we’ll one day look back on the 2020s epidemic of mental illness as we now look at cholera outbreaks— something that came with a particular stage of modernization, but as its source became better understood, became increasingly manageable and preventable over time.

Some good news (for both counselors and the public in general)

There is evidence of a real improvement in mental health so far in 20244, whether due to effective counseling, the shock of social media on well-being5 wearing off, or even the possible addiction-reducing properties of Ozempic. That we saw more demand for counseling in 2024 even as overall mental health appeared to improve suggests that a shift in culture might drive growth in this area going forward. There is a big disparity in the prevalence of mental health issues across age groups, and over 50% of Gen Z having received professional mental health services at some point. With therapy of this nature so normalized, it wouldn’t be surprising to see increases not only in mental health counseling but also in marriage, career and other forms of counseling, especially if technology accelerates changes in daily life. With this in mind, it’s also worth noting that worries about the future are the #1 source of anxiety for Gen Z— and worries about finances are not far behind. This is all the more reason to work on more accurate job forecasts!

The forecast

Taking all of this together, it seems like 3% annual average growth over the next 10 years seems like a nice middle-of-the-road estimate between a continuation of current trends and the possibility that better technology or circumstances might reduce the need for counseling. That’s a 34% total increase, or 230,000 more jobs than the official forecast!

23,000 new jobs per year is slightly more than the latest graduation rates of counseling programs. But enrollment could increase in the future, or the demand could be filled by a mix of professionals from other fields like psychiatric nurse practitioners or in industries where educational requirements might be loosening such as religious6 and nonprofit social service organizations7. The strong connection between exercise and mental health might also open up new avenues of treatment.

A 34% increase in mental health counselors might mean going from 13% of Americans regularly seeing a professional to over 17%8 — much closer to current needs and possibly even fully meeting them if rates of mental illness continue to fall. By the 2030s, some of the more complex problems in neuroscience mentioned above may have also been solved, further contributing to “cognitive freedom”. In such a world, the human element of counseling may evolve with the increased number of possibilities, but I predict that it will still be just as important.

Management

Researchers in the field of AI safety often speak of the “alignment problem”: given sufficiently intelligent AI, how will we ensure it can still help us produce the things that we value?

While there will be many technical details to work out in the years to come as AI systems are designed to operate computers and integrate more deeply into workflows, a big part of the solution might be an old school one— more empowered managers.

Even in fields like software development that started seeing a slowdown in hiring in 2022 and AI-enhanced productivity since then, the growth in managers has been robust.

This makes sense, for a variety of reasons. Where new technology improves productivity, prices usually fall, and more complex products and services can become affordable. But as production processes change and/or the scope of work expands, the complexity of decisions that managers need to make can also increase.

Now, this may be less true in fields where the products or services produced are simple and unlikely to change much. But earlier waves of automation in agriculture and manufacturing have left relatively few jobs in these industries. In fields from science to logistics to construction, one can imagine how complementary intelligent systems (including increasingly capable robots) could further increase the complexity of what is produced and require more management decisions, including when to deploy artificial vs. human intelligence. There is already research to show how managers are often more effective users of AI than younger “AI natives”.

A counterpoint to this is where AI has been employed in automated reporting up to leadership, cutting out layers of middle management. But here, I think it is helpful to draw a distinction between managers and supervisors. Where the number of human individual contributors shrinks, the number of first-line supervisors likely will as well. But just as some have hypothesized that AI tools might empower more “solopreneurs”, greatly assisting them with sales and customer service functions, an increasing number of individual contributors in larger organizations might oversee enough automated and hybrid processes to be considered managers as well, albeit with fewer human direct reports.

A helpful way of seeing how this trend is evolving over time is by looking at the ratio between managers and line level employees across different fields (as reported by BLS):

The ratio of managers in sales, marketing, advertising, and public relations relative to other sales workers, and how much it’s grown— up 50% since 2016(!) is quite striking. While part of this story is the 2010s decline in retail, the number of managers has actually been rising even faster. It’s also worth noting that marketing and public relations teams have some of the highest rates of AI adoption. The departments that are making the heaviest use of AI could end up becoming the more influential ones in their companies.

Supporting the theory of increasingly complex work requiring more managers, we also see a notable rise in the ratio of managers to line-level employees in science and tech. But suppose we invert this chart to show how many line-level employees there are per manager, and also include BLS 2033 projections?

The number of employees per manager is quickly falling in many sectors—administrative services and transportation, storage, and distribution are the most noticeable ones, but we see a significant drop in this ratio in the fields of industrial production and medical/health services too. Surprisingly, the BLS doesn’t forecast any further change in this ratio across most fields; this is another area where I suspect these projections could be improved. A future where AI has advanced considerably could see the number of employees per manager in some fields fall to near one, as it is in agriculture.

Among the sectors where the ratio of employees per manager is falling the quickest, circumstances vary quite a bit. In administrative services, which includes customer service and secretarial work, it’s because the number of line employees is falling significantly, but the number of managers is increasing slightly. In transportation and distribution, it’s because growth in management has outpaced even the unexpected increase in warehouse workers. As robots in warehouses become more sophisticated, we might expect to see more line level-employees shift into management.

Looking at charts like the one above, it seems like the collapse in the employee-to-manager ratio is just getting started. At the very least, I think we can expect to see 1.5% growth per year— similar to the trend since 2018, but taking into account slower growth in the labor force overall. That would be roughly 3.5 million more jobs by 2033.

The bottom line

Between ~70,000 more life scientists, 230,000 more counselors, and 3.5 million more managers, we’re looking at 3.8 million more jobs overall! This is substantially more than the 2-3 million additional jobs we might expect to lose due to technological displacement from telemarketers, customer service reps, and administrative assistants. However, we need to be careful here. Most of these new jobs require either extensive education or experience which won’t be as easy for laid off workers from other fields to pick up. Even more importantly, the growth in managers may come mostly from promotions and reclassifications, meaning effective declines in other occupations. So it is still highly possible that we lose jobs on net.

The circumstances here are consistent with something I’ve observed before: the job market might continue to be relatively decent for experienced workers, but increasingly tough for young people, especially if updates to school curricula lag behind the knowledge and skills needed to be successful in increasingly AI-augmented disciplines. While unemployment has ticked up a bit in 2024, it is noticeably worse for those under 25— a problem that newly-elected governments both at the federal and state/local level will have to keep a close eye on.

“Learning to code” was common advice for young people and career switchers alike in the 2010s. But is it such good advice today? Or is it better to learn a trade?

Over the coming weeks, I’ll answer these questions while taking a look at a few sectors where job prospects are a little more uncertain and could swing the economy one way or the other.

7

A more detailed breakdown of Substance abuse, behavioral disorder, and mental health counselors shows a strong increase in employment across all major industries. Below is the 10-year moving average of annual growth of each industry that employed a significant number of counselors, including the latest BLS 2023-2033 projection:

It seems unlikely that the growth rate would fall so significantly in every industry. Averaging 3% per year through 2033 seems like a much more reasonable (and even conservative) assumption.

It’s easy to dismiss the aggressive forecasts of pernennial techno-optimists (and doomers) as more of the same hype that has failed to materialize. Even their more in-depth explorations of this question often evade what specific numbers for particular occupations might look like over time. Meanwhile, official government forecasts like those from the BLS, as detailed as they are, can be unrealistically conservative, underweighting both historical trends in some cases as well as the potential impact of new technology.

Taking into account some lessons from predictions that were made almost 10 years ago, I’ve tried to bridge the gap between the hype and the realities of how complex the world truly is. If you or someone you know is considering future career paths, here are three jobs to avoid:

Telemarketers

Robocalls may finally be on their way down, but the number of telemarketers has been declining since at least 2004, down to just 1 in 2,000 workers in 2023. Based on current trends, this occupation looks set to join “elevator operator” and “motion picture projectionist” as a relic of the past by the end of the decade.

Of course, I doubt there are many people out there who consider telemarketing to be their calling. It is one of the least satisfying jobs out there. I’d be shocked if 66,800 Americans were still working in this field in 2033 as the BLS projects, given the historic trajectory. Even in 2020, only 19% of Americans answered unknown callers, a number that has surely dropped since then.

Customer Service Representatives

In May 2024, the Swedish financial technology company Klarna made waves by reporting that its AI assistant was able to handle two thirds of its customer service chats1. This is obviously a lot quicker than the 5% decline in customer service jobs predicted by the BLS.

Losing this many customer service jobs across the entire economy within the next year or two would be a nightmare scenario affecting two million Americans and would bring up unemployment by a whole percentage point on its own. Fortunately, change this fast across an entire occupation would be entirely without precedent and is extremely unlikely to occur so quickly. But the recent 5-6% annual losses in the business support services industry2 suggest it’s possible that we could get there sometime in the 2030s.

However, this is a worst-case scenario. Other industries have had better prospects, at least up to 2023:

Whether this same demand for US-based, human customer service workers will still be there in the 2030s is the key question. On one hand, 2023 BLS projections note that growth in healthcare is likely to continue due to “federal payment models emphasizing patient satisfaction”. But better software (which AI is likely to make cheaper) could also reduce the need for customer assistance, even in highly regulated industries and government where the adoption of labor-replacing chatbots and other AI-powered technologies might be slower.

This dramatic downturn in customer service representatives is already visible in Indeed.com job openings data, which has proven itself to be a moderately reliable forecaster of employment in this sector. Openings are down over 20% relative to the job market as a whole since early 2022 and continue to drop.

Unless we see a reversal in this trend, it’s reasonable to expect a 20% drop in customer service jobs over the coming decade at minimum. But at least 30% or more seems likely if we see technology further progress. If that comes to pass, it would mean roughly a million American workers, or 100,000 per year (6x more than the BLS expects) who if not near retirement could be looking for a new line of work.

Secretaries and administrative assistants

The number of secretaries and administrative assistants has been in decline since 1980. But the BLS’ projection for 2033 is quite remarkable: they projected a slight increase! This was one of their biggest revisions from the year before.

I think this is wrong. But I can understand why they were optimistic. From 2020 to 2023, the number of secretaries actually held steady, breaking the ominous trend that was apparent in this job in the late 2010s.

This might have been in part because job losses here were so steep in 2020 that layoffs were effectively “pushed forward” a few years. Indeed, we even see job openings in administrative assistance rise above the overall market in 2021 all the way through 2023!

That trend appears to be over though. October 2024 has started with another ominous drop in job openings in this field, and I suspect as a base case that we will return to the familiar decline in administrative work we’ve seen since the 1980s—at least another 20% drop, but possibly as high as 40% by 2033.

This would impact roughly a million workers, or even more if we include similar underestimates of the potential for further automation in other clerical occupations like billing, bookkeeping, accounting, and auditing clerks, as well as other smaller administrative occupations. There are even some second-order effects in other areas to consider, such as if fewer HR personnel or managers are needed.

The only bright side is that we’re unlikely to see a big wave of tech-induced layoffs in this field. The expertise that experienced administrative assistants accumulate about their organization over the years might make them safer from layoffs than other types of workers. The decline might be driven by retiring workers simply not getting replaced, or their duties instead getting distributed to other types of workers like “analysts”. The clearest evidence we have for this is that the average secretary or administrative assistant is now much older3 than the average worker.

I think this is an important trend to consider in an era of dramatic predictions about AI. The reliability of an experienced administrative assistant (and one who might make use of AI tools when appropriate) might win out against an untested AI “virtual assistant” for some time to come. But the calculus will surely be different when it comes to making a new hire.

The bottom line

Our fear of dramatic shifts in employment might be too informed by the manufacturing sector in the 1980s and 2000s, where plant closures eliminated large number of jobs all at once4. The customer service and administrative jobs most threatened by AI are in a much broader array of industries, each with their own unique considerations which reduce the odds that any single development will cause a lot of technological unemployment. In the absence of complete closures, the impact of layoffs on morale might temper job cuts compared to earlier dramatic declines in manufacturing. This might bring about a slightly softening labor market, with hiring at recession-era levels but layoffs still low, similar to what we are seeing today. Might we see a deepening of “The Great Stay”?

If so, then the most apparent threat is to younger people. Sales, customer service, and administrative assistance have historically been common entry-level jobs. As these opportunities evaporate, the uptick in youth unemployment has been notable and something to keep a close eye on. Combined with disruptions to education from pandemic-era school closures to curricula unprepared for generative AI, the next generation of job-seekers might be further disadvantaged even if more experienced workers manage to avoid tech-induced layoffs.

But just as technology destroys some opportunities, it creates new ones. Next week, we’ll take a closer look at a few jobs that are set to grow even more than the BLS is predicting. Can these make up for the nearly 3 million jobs we might lose by 2033?

Their projections tend to be most accurate in the healthcare sector, where a predictably aging population keeps steadily increasing the demand for professionals like registered nurses.

Forecasting business cycle change is somewhat more difficult, but given that their projections are designed to reflect an economy at full employment, their estimates for occupations like construction equipment operators have been on the mark lately, after adjusting for the unusual2 amount of demand for construction labor during the 2000s housing bubble. However, they did miss the unexpectedly large and unforeseeable growth in operations and other managers3 that came with the post-pandemic surge in entrepreneurship.

Forecasting the future of occupations where employment is subject to politics and popular opinion also has a mixed record. BLS projections for occupations like K-12 teachers and police officers are usually neutral relative to labor force growth. This can sometimes a good bet for these steady primarily-government jobs, but not always. There was a significant decline in teachers in the early 2010s, and an ongoing steady decline in police and librarians.

However, it is technological change that is notoriously hard to predict. There were three big stories here between 2014 and 2024:

• The rise of the data scientist and related occupations

• The demise of retail sales

• The steady decline of secretaries and other office clerks

Could these changes have been reasonably predicted, or were the impacts of some emerging technologies truly “out of the blue”?

Let’s take a closer look.

The rise of the data scientist and related occupations

The growth in math-related4 occupations since the late 2010s has been truly impressive. While the BLS correctly projected growth here, they only estimated that these types of jobs would beat the overall market by 23%. In reality, they crushed it by an incredible 553%!

A good thing more people are finally making use of all of the math one learns in school right?

All kidding aside, could the BLS have realistically anticipated something like this happening?

Looking at Google search terms, the demand for these occupations was foreshadowed by a significant interest in “big data” in 2011-2012, which steadily grew through 2015. The term “data scientist” was also on the rise, after first being coined in 2009 alongside a prediction by Google’s chief economist that these sorts of jobs would be the “sexy” ones to have in the 2010s.

On the other hand, hindsight is 20/20. There wasn’t much sign of a dramatic increase in this occupation at this point. Looking at 2014 and previous figures and projections, even the 23% increase was quite a departure from previous forecasts.

In fact, even if they had taken Varian’s prediction that statisticians (and other related mathematical occupations) would grow as fast as IT-related ones in the 1990s5, they would have only been about twice as aggressive as they already were, not the 20x+ more they would need to have anticipated the full increase. Predicting 23% growth was quite reasonable given the totality of the information at the time.

We tend to think of the 90s as a revolutionary decade in terms of computers coming into everyday life. But perhaps this is because this was the decade where many workers outside of the IT field began using computers at work for the first time. Growth in IT-related jobs was considerably modest compared to the truly unprecedented percentage growth in math-related occupations in between 2018 and 2022.

It’s also worth noting that there were a few other closely related jobs that grew alongside these statisticians and data scientists, though not as dramatically:

Market research analysts, marketing specialists, and survey researchers: often tasked with producing the large amount of data processed by data scientists and statisticians, this occupation has grown steadily since 2004 and has consistently been under-projected by the BLS6.

Other business operations and management specialists7: these workers, while loosely defined, include many product and project managers whose decisions are often shaped by the greater availability of data. Growth in this field has followed a very similar trajectory to that of data scientists.

Data scientists and related occupations: the bottom line

All together, the growth in these analytical professions contributed significantly to the exceptionally robust job market of the late 2010s and early 2020s that skewed much more toward these growing fields than other declining ones. But how predictable were some of the most notable declines?

The demise of retail

In the 2000s, the BLS predicted the decline of certain production occupations such as Textile machine setters, operators, and tenders8 exceptionally well. But of all of the jobs that declined between 2014 and 2023, their projection for retail salespeople was their biggest miss9.

Looking at this chart, we can see exactly why. Predicting steady employment in this occupation was a good bet all the way through 2014! Only in 2016 do we begin to see a drop. So was this just another surprise?

The curious thing about this decline is that there is another occupation that is nearly a mirror image of it: stockers and order fillers.

These workers are on the front lines of the warehouses at Amazon and other online retailers. But as late as 2012, the BLS was instead projecting relative declines in this occupation! 2014 was the first year their projection was even neutral. But just like how 2016 saw the first decrease in retail salespeople outside of a recession, 2016 saw a similar increase in stock clerk employment. Could the BLS have reasonably predicted that this growth would effectively come at the expense of brick and mortar?

Amazon’s history is very instructive here. In the summer of 2015, it wasn’t necessarily obvious that online retail would have continued expanding in the way that we’ve since observed. The company, which wouldn’t vault into the top 20 until the next year, was still famously unprofitable at the time, and its sales in North America were notably slower in 2014 compared to previous years. Even Google searches for Amazon had flattened while searches for eBay actually declined. It might have been reasonable to assume we were nearing the top of the curve in terms of interest in online shopping, at least in the US.

There were only a couple of hints that online retail was set to reshape the job market. One clue was Amazon’s valuation: by Q2 of 2015, it had doubled its market cap relative to two years ago, even though revenue only increased by 47%, and earnings per share were still hovering near zero. Investors were clearly making a big bet10 on the stock.

Of course, investors sometimes make bad bets. It would be unreasonable to expect a government projection to match the enthusiasm of some investors alone, especially if the labor market impacts of a shift to online retail weren’t yet fully appreciated. But there was one other clue that the BLS should have picked up on:

The number of retail salespeople reported by the Census ACS was already starting to fall as early as 2013. However, BLS OEWS numbers did not catch this, and it seems like the Employment Projections team may have trusted their numbers over the Census’. But why was there such a difference in the first place?

Differing employment estimates explained

Early reports of Amazon’s impact on retailers came primarily from smaller businesses like local bookstores who are likely under-represented in BLS’ surveys of employers. The threat to these small businesses, both from big box competition as well as online retail, was well established in the public consciousness too— “Small Business Saturday” had been a thing since 2010! These small businesses also employed considerably more people relative to their revenue. The workers who lost their retail jobs during this early period of Amazon expansion would still get picked up by the Census years before the “retail apocalypse” that sent many larger retailers towards the bankruptcies and mass layoffs and made this trend more apparent.

But wait, if BLS numbers aren’t counting some of the workers picked up by the Census, why does BLS record a higher number of workers?

This is another key difference between Census individual surveys and BLS employer-based ones, and a good example of why it’s important to get information from both. In the case of individuals with multiple jobs, the above Census employment numbers would report how they described their work in general. A worker who had multiple retail jobs would only be counted once in the Census, but multiple times in the BLS, since the BLS asks each business to report all of their full and part-time workers. To give another example, an independent musician who also supplemented their income by working at a department store might be counted as a musician by the Census, but only as a retail sales worker by the BLS, because their independent work as a musician would not be accounted for by an employer.

The demise of retail: the bottom line

It’s easy to get lost in the differences between the various government surveys, but I do think a more aggressive prediction for a decline in retail sales work would have been well-justified based on the exceptional drop recorded by the Census in both 2013 and 2014. I call this exceptional not because it was exceedingly large, but because in these years there was a notable economic recovery which is typically good for retail jobs. That the percentage of workers in this field continued to decline in spite of that, falling below even 2009 Great Recession levels should have been a warning sign. While there was some evidence to suggest that the market for online retail was already saturated by the mid-2010s, there were credible arguments (later proven true) that this trend still had plenty of room to run. I think it’s important for economists to not be too hasty to call a bottom or a top where there is still potential for significant movement. Offering a range of uncertainty would be an interesting feature to add to BLS employment projections in the future.

The steady decline of clerical work

While the decline in retail sales jobs was something new in the 2010s, the relative decline in secretarial and other office and administrative jobs was something with a long history going all the way back to the 1970s and 80s.

As you might guess, the BLS projected very little change here. While this might have been reasonable for other office and administrative support workers11, it seems like they were predicting a bottom in the number of secretaries and administrative assistants12 that ultimately didn’t pan out.

Given the long history of a steady relative decline in this occupation, it seems like a claim that this trend would stop all of the sudden should have been supported with more evidence. Indeed, this seems like sound advice for the BLS to follow today! While they are predicting a slight decline in legal and other secretaries, they predict enough growth in medical secretaries to bring the overall occupational group back to near neutral.

This would again be quite exceptional given the previous longstanding trend. I would need more evidence to believe that we have hit any sort of floor in the number of secretaries, especially with demonstrable progress in AI virtual assistants. While such tools will need further refinement and adoption might be slower in more regulated industries, it would surprise me if there was any slowdown in the software-assisted decline in administrative work. And if AI makes good software much cheaper to produce as some are predicting, the accelerated pace of application development would make up for any slowdown due to easier problems already having been solved, at least for another decade to come.

While the BLS’ prediction of a decline in secretaries and administrative assistants is in my opinion unreasonably conservative, they are predicting that productivity change will significantly reduce the number of other office and administrative support workers in a way that they didn’t for 2014-2024. This hypothesis is well supported because the decline that this occupation saw during the early computer era from 1970 through 2004 has resumed.

Given the consistent decline in other administrative occupations, and the overall trend of general occupations becoming more specialized, even this 10% relative decline may be too conservative. If the overall record of occupational change is such that nearly 30% of workers have historically been in occupations declining by more than 10%, are there really that many jobs that are more likely to be automated or specialized away than this one?

Can we build a better jobs forecast?

With these three examples in mind, how could the BLS do better next time?

I have four major takeaways:

#1: Include visuals of past job growth and projections.

Sometimes, a good visual can reveal a trend that would have otherwise been hidden behind the abstraction of multiple complex tables (which is where historic BLS projections data lives now).

While there may be some complications in showing the trajectory of occupations whose definitions have changed over time, this only has a significant impact in a few fields after 2000. Getting these visualizations right also makes my next two takeaways easier to realize.

#2: If a predicting a departure from a previous trend, the justification should be strong

As we saw when looking at administrative work, the tendency to make a conservative projection sometime flies in the face of longstanding historic trends. Better visualizing the historic trajectory, and remembering how dynamic the job market has typically been should help correct this problem and allow for more appropriately aggressive forecasts.

#3: Estimate a range of uncertainty

Sometimes the beginning of a new trend, or the end of an old trend is in sight but hotly contested. Or maybe there have historically been wild swings in how many people work in the field. These are all good things for job seekers to know about! The unpredictability of a particular occupation is good information to have.

#4: Pay extra attention to large occupations.

As an individual who does this analysis in my free time, it would be quite difficult for me to give equal attention to all 832 different occupations, some of which are really small. If the BLS had correctly guessed just half of the actual change that happened to retail salespeople, stock clerks, and secretaries, they would have beaten a naive projection by an additional 1%, nearly doubling on the 1.3% they scored for their 2012-2022 projections. This impacts a lot of people! By improving the forecasts, particularly for large occupations, government at all levels may be more prepared for any possible technological displacement to come.

Conclusion and questions for next time

Is it possible that the BLS has made significant improvements to their forecasting since 2014?

Only time will tell. It’s worth noting again that their recent projections are even more conservative than their historic ones. If their default way to indicate uncertainty is to make a neutral projection, does this mean that there is even more uncertainty now than there was 10 years ago?

Since 2020, the BLS has also published some detailed factors that influenced their predictions. Next time, we can take a look at if any of the factors indicated have been more correct in forecasting change. We can also take a close look at which of their recent projections have changed the most when compared to last year. What impact, if any, has AI progress had on them? And what trends are they potentially missing?

2

The jump in construction employment during the 2000s housing bubble is quite striking over the course of the 152 years of data that the Census has about these occupations!

3

The growth in general and operations managers was exceptionally striking, and corresponds with a rise in new business formation.

It’s important to note though that not all of the growth was from entrepreneurs: government was the industry that saw the greatest percentage increase!

However, growth was quite robust across most industries.

6

12% projected growth since 2014 with 67% actual growth above employment realized:

7

My references to Other business operations and management specialists aggregate the following BLS occupations over time:

• Emergency Management Directors (11-9161)

• Emergency Management Specialists (13-1061)

• Project Management Specialists (13-1082)

• Business Operations Specialists, All Other (13-1198 and 13-1199)

As seen below, there was dramatic growth in this occupation between 2018 and 2020, even after making some adjustments to account for changing occupational definitions:

8

I think this is one of the more beautiful illustrations of an accurate BLS projection, though there are undoubtedly some heavy stories when job losses are so dramatic, even if they were predictable:

BLS has regularly published evaluations of their previous projections. In their most recent analysis of their 2012-2022 projections, they find that they correctly predicted whether or not an occupational group would grow or decline 77% of the time.

This sounds pretty good until you take a closer look. In an economy where the population and workforce are growing, it’s somewhat easier to get the overall direction right because a majority of occupations are likely to grow nominally. A more challenging test would evaluate how many occupational groups were correctly predicted to grow faster or slower than the whole workforce. The BLS does this too, and for 2012-2022, they were right 67% of the time. Knowing that a random guess should be about 50% right, their accuracy is only about one third of the way between a random guess and a perfect prediction.

However, knowing the direction is only one part of making an accurate prediction. There is a world of difference between an occupation where employment only declines by only 1% and one that sees a dramatic 50% drop. The metric that the BLS uses to evaluate accuracy in this regard is a comparison of the sum of absolute differences. In other words, they compare which prediction is closer: their detailed one, or if they had instead assumed all occupations would simply grow with the size of the total workforce. For 2012-2022, their sum of absolute differences was 17.1, slightly lower (and better) than the naive (default) assumption of 17.8. This metric isn’t a technically expressible unit, but it is similar to how one might measure the total percentage difference between the original and forecasted mix of jobs vs. the actual mix for the forecasted year.

A few months ago, I wrote about how this percentage could be a more precise way to measure how much the nature of work has changed in each decade going back to 1870. I used the same approach for BLS employment projections back to 2004-2014: measuring how much aggregate change actually happened2 for each set of projections, and how much the forecast beat a naive (default) assumption that employment in each occupation would remain the same over the next 10 years.

The BLS does a little better using this metric, with a 1.3% difference in 2012-2022 rather than 0.7, but there is a worrying trend: BLS forecasts are inching closer towards being less accurate in the aggregate compared to the default assumption of employment remaining completely unchanged. And while we might have expected the 2010-2020 projections to have been off the mark due to the pandemic, the 2012-2022 ones are actually even further off.

Of course, this doesn’t mean that we should just assume little change out of an abundance of caution, or that it is impossible to predict with any accuracy where jobs will grow or decline next. Indeed, a key flaw in BLS occupational projections is that they are far too conservative! I have alluded to this in some of my previous work, but I’ve finally done a more complete analysis here. If we imagine how much occupational change the BLS forecasts estimate vs. how much happened historically, we see that their forecasts only capture about one third of the actual change (and possibly less heading into the 2020s).

That said, when the BLS does make a bold prediction about a particular occupation, it’s often right. This has been particularly true of predictions about healthcare-related jobs, which makes sense given reliable demographic projections. This is perhaps most readily seen in a comparison of bell curves between 2012-2022 projections vs. actuals.

On the other hand, the BLS’ most consequential miss of the 2010s was underestimating the impact that the rise of online ordering would have on the job market. This is a major component of the rise in “Transportation & moving” occupations3. Relative to employment growth, stockers and order fillers were predicted to decline by 11%. Instead, they increased by 17%. On the other hand, a 1% decline was predicted for retail sales workers, but this decline actually amounted to 19%! Correctly predicting growth in these two occupations alone would have improved overall accuracy by 1.35%, doubling it relative to the naive status quo prediction.

Based on the information available in the summer of 2013 when the BLS made these 2012-2022 projections, could they have realistically anticipated how much online ordering would shape the job markets in retail trade and logistics? To what degree is the evolution of technology and its impacts on the job market inherently unpredictable? How might recent advances in technology have shifted this years’ predictions relative to last years’, and what advances are being overlooked?

I look forward to digging into these questions over the coming month!

One indicator we can rely on: before people are laid off, companies usually cut back on hiring. The Bureau of Labor Statistics’ Job Openings and Labor Turnover Survey (JOLTS) captures job openings, hires, quits, layoffs, and other separations. It’s typically published a few days before the Employment Situation (AKA the monthly jobs report). The catch is that it only includes the previous month and is broken down by broad industry. So as of late August, we have July employment numbers by detailed industry, but our latest indicator of where job openings and hiring stands is from June.

Fortunately, we have another data source this is regularly updated on a roughly weekly basis: Job Postings on Indeed.com, both for the United States and several other countries!

As expected, the job market has cooled from the heady days of the pandemic recovery, with job openings at their lowest levels since April 2021. Yet July 2024 saw a small rebound the likes of which haven’t been seen in over two years. Might this give us some optimism?

While there are some worries about fake job postings distorting the average, it has been a good leading indicator of the number of job openings reported by JOLTS these past few years. And even more importantly, since 2022, the level of job postings1 on Indeed has reliably forecasted employment growth 6 months out!

A scatter chart reveals the true strength of this relationship. If this trend continues to hold, we might see some better-than-expected jobs reports this fall.

If this indicator isn’t getting the attention it deserves, it may be because of its history. It is a fairly new one, which wasn’t published until February 2020, which was… rather unfortunate timing. While it remained a good signal of how much interest there was in hiring throughout the pandemic, the course of the pandemic was much more indicative of how the actual job numbers would change in future months. Even during the recovery in 2021 and early 2022, so many workers quit during the Great Resignation that the number of job openings went up even as growth in employment fell in some sectors. The food service sector is a great example of how the relationship between job postings and employment growth can actually reverse itself in a very hot labor market.

Indeed offers a breakdown of job postings for 30 different sectors. 19 of them appear to be at least slightly predictive of job growth 3-6 months in the future, and 11 of them are at least moderately predictive!

One of the strongest and most forward-looking correlations has been between the level of job postings in healthcare and growth in healthcare-related industries. These closely tracked each other from October 2020 to early 2023, but we are are now at a critical juncture: if the level of healthcare job postings from 15 months ago is still indicative of today’s employment trends (presumably due to longer hiring timelines in this industry), we should see job growth in healthcare-related industries cooling off very soon.

That we can with some reliability see a few months into the future of dozens of broad occupational groups might give us some clues into early signs of AI and other technologies that are disrupting work. With some companies claiming that their customer service chatbots are already handling the work of hundreds of employees, can we see this trend in Indeed’s customer service job postings?

Yes! Linear regression analysis confirms that the level of job postings is at least slightly correlated with the declining employment 3 months later in the Business Support Services industry which we explored last time.

The impacts of new technology are nearly as difficult to forecast as changes in politics and the financial markets. But prices, whether for shares of stock, or shares in a prediction market are a signal of where the rubber meets the road. Significant increases or decreases in job postings are a good real-time signal of companies’ commitments to hire a few months into the future. While it is important to recognize when the usual rules no longer apply, the advantage of being in a steady-state jobs market is that we should be able to see the signs of new technology reshaping the economy a few more months in advance than we might have otherwise. This can make all the difference to those who are making longer-term forecasts.

Having better information about the jobs market ahead can help those who are starting their careers (or thinking about a switch) make a more informed decision. Catching a trend just a few months earlier can have a big impact. The Bureau of Labor Statistics’ 10-year projections, which are used by many job seekers and guidance counselors alike, have historically struggled with forecasting the impact of some new technologies. We’ll take a closer look at their 2033 forecast when it comes out next week, along with a deeper dive into how these projections have performed in the past.

But if new technology were to accelerate our pace of change all of the sudden, what would be our first clue that it was really about to happen?

We might see some dramatic anecdotes from the news, or even personal experience. But it can be hard to get the full picture from this. After all, sorting out hype from reality in the world of AI these past few years is a challenge we’ve all had to deal with. If robots or AI were about to take our jobs, what are some hard indicators that might catch them in the act?

The detailed occupation figures from the Census and BLS that I’ve cited in the past won’t cut it since they are published only once a year with a significant delay. Spring 2023 data is old news by the time it is finally released between April and September of 2024. However, detailed industry data is available with the Employment Situation every month, and we can even measure the degree of change with a similar index that we used before to see if changes in jobs are happening economy-wide or only in specific industries.1

We see a very significant uptick in change to the overall composition of employment in July 2024— 0.32%, or nearly 40% than the previous more normal reading of 0.23%. This is the highest we’ve seen since February 2023. And prior to the pandemic, the last time this indicator was as high was in January of 2012! It’s worth noting that the decline in retail jobs from 2012 to 2019 (one of the most significant and easy-to-identify casualties of technological and consumer change in the 2010s)2 didn’t cause this indicator to rise to this level, so something significant must be happening.

Taking a closer look at the biggest declines of July 2024, we can see how technology is just one of three major factors driving change in the job market today.

Technology-driven change

The clearest example of these declines that is likely technology-driven is that of business support services. After peaking in late 2016, employment in this industry, which includes a significant number of customer service jobs threatened by AI as well as outsourcing, has steadily declined. July 2024 has seen an acceleration of that trend.

The non-internet publishing industry has been another clear casualty of technological change. After a dramatic decline in the late 2000s, it spent the 2010s just below business support services, before surpassing it again during the pandemic.

Since late 2022, it has returned to a declining trajectory. That said, this decline was not as steep as its losses in the 2000s, and July 2024 was in line with previous months, similar to jobs elsewhere in the broader information industry. And this industry as a whole is shaped not only by evolving technology but also the cooling effect of higher interest rates.

Business cycle-driven change

The trend in employment in rental and leasing services is eerily similar to what it was in the summer of 2008. This reflects the rise in vacancy rates which increase before and during recessions.

The good news is that total employment and vacancies are roughly where they were pre-pandemic. If these trends slow down with the expectation of lower interest rates, they could be good leading indicators for a “soft landing” (or not).

Policy-driven change

Nearly one in six jobs lost in July 2024 came from an unusual decline in employment in local government. However, this was more than offset by an increase in jobs in local government education. As the early 2024 wave of migration settles into more permanent housing, does this reflect municipalities shifting priorities and adapting to the increased demand anticipated at schools?

Industries influenced by a mix of factors

It’s rare that any one factor explains everything, but 1 in 10 jobs lost in July 2024 were in transit and ground passenger transportation. These losses could be plausibly explained by all three factors we explored above.

This industry saw an exceptional 2.5% overall decline in jobs in one month alone. With Waymo significantly expanding its service in 2024, are we beginning to see some displacement from self-driving cars?

10,000 jobs lost seems a little high to be explained by just Waymo (estimated at over 50,000 driverless rides per week). It’s possible that a cooling economy is driving a decline in demand for taxi drivers, or that the decline is just random like the one seem in April 2023. It’s even possible that the nearing expiration of IIJA funds for transportation in September 2026 is causing some cuts to local transit authorities in advance of that as well.

Whatever the case may be, this is a sector to watch closely for a variety of reasons.

Looking on the bright side

After taking a deep look at declining sectors of the economy, it’s important to remind ourselves that even in this exceptional month, employment in growing industries still outpaced the declines by 2-to-1. Continued growth in healthcare and construction employment have been the backbone of this labor market. And while accomodation and food service employment isn’t growing like it used to, it is still a significant positive contributor to labor demand.

Meanwhile, growth in delivery-focused industries such as warehousing, couriers, and other support activities for transportation exceeded expectations and appear to be on the upswing again.

The strength of these sectors even amidst other warning signs in the economy speak to the relative strength of the consumer. But there is one more indicator that is increasingly proving itself as even more forward-looking than these monthly numbers from the Employment Situation. This one might allow us to peek 5+ months into the future of work. We’ll take a deeper dive into it next time!

2

I should also note that the decline in employment in retail trade as an industry (tracked monthly) was not nearly as severe as the decline in retail salesperson as an occupation between 2012 to 2019. I hope to explore this distinction in a future piece.

These worries actually go back much further than most people realize. We see fears about jobs (and intellectual property rights of artists) going all the way back to the invention of recorded music. Why go out and see a live show when you can listen to all the hits of the day in the comfort of your own home on your Edison Phonograph?

Indeed, the number of musicians dropped by nearly 25% over the 1910s when this cartoon was published. In addition to the explosion of record sales, Prohibition and cabaret taxes further devastated the live music industry, which by 1920 was bouncing back from the Spanish Flu epidemic of 1918.

Recording and transmission technology advanced even more in the 1920s. Improvements in sound quality led to musicians losing jobs in theaters when “talkies” first hit the scene in 1927, and radio allowed an even cheaper medium for access to a greater variety of music in ways the foreshadowed the impact of digitization and streaming nearly a century later. But in spite of these developments, the Census shows that most of these musicians were still able to make a living, as the total number actually started to grow again.

What can we learn from the last time we were in the 20s and technology was rapidly reshaping the music industry? Several factors might have helped the musicians of that era:

• The increased diffusion and popularity of a variety of musical styles (notably jazz)

• More economic prosperity and leisure time enjoyed by workers

• Bold actions taken by the American Federation of Musicians (AFM)

Technology has continued to evolve, and the revenue made from recorded music has fluctuated significantly with economic conditions and the ever-changing form factors.

But the percent of the workforce that makes a living from music has remained remarkably steady ever since: about 1 in 750 workers, or 1 in 200-3001 if you include art, drama, and music teachers, who were counted in the Census occupation “Musicians and music teachers” prior to 19702.

It’s worth noting that the above chart only goes up to 2022. Looking more generally at jobs in the performing arts, spectator sports, and related industries (which also includes the many other types of professionals needed to put on shows), we see over 100,000 more jobs— a 20% jump— in the 16 months between January 2023 and May 2024 alone. This unprecedented increase in interest in live events, which had already fully recovered from the pandemic by early 2023, has even been identified as a significant contributor to inflation.

What takeaways from the past are still relevant today?

There are some notable similarities and differences between the 1920s and 2020s. Like the first diffusion of records in the early jazz era, streaming has exposed people to a greater variety of music than ever before. Between younger people preferring experiences over things and shorter working hours than previous generations, as well as a burgeoning population of retirees getting out to enjoy events like “Geezer Happy Hour”, there is more free time available to get out and have fun.

On the other hand, old music is an increasingly large share of what people listen to; a trend that’s likely to continue with an aging population. This might make it harder for new artists to gain traction, but the internet-accelerated trend of micro-culture over mass culture helps support a broader base of artists too. But does micro-culture make it harder to unionize? Although the number of musicians has grown with the population, membership in the AFM fell through the 2010s and in 2024 is less than half of what it was 100 years before.

The bottom line

While advances in AI could threaten artists’ income made from recorded music, it might also complement human creativity and lead to new styles similar to how photography influenced the development of impressionist art. Fundamentally, it seems unlikely to depress turnout at live shows in the way the phonograph did in the 1910s. And income from shows (along with merchandise) comprise the lion’s share of most musicians’ income.

However the average earned income for musicians has barely moved up at all since 1990 in inflation-adjusted terms, hovering at just over $40k per year ($43k in 2022)3. This stands in contrast to overall average earned income which has grown from $51k to $63k.

That said, it’s interesting to compare how many musicians there are against other creatives. In 2009 there were almost equal numbers of musicians, visual artists4, photographers/videographers, and authors. But all of these other occupations grew faster in the following years.

Given the development of the internet and social media over the 2010s, it’s not surprising to see growth in Etsy creators, Youtubers, and bloggers outpacing musicians. But with people starting to spend gradually less time online for the first time ever, it might be about time for a renaissance of live entertainment. As clever as AI music generators might get, it’s difficult to imagine them replacing the desire for human connection that drives people to see their favorite artists in-person.

P.S. For more thought-provoking reads about music and culture, I highly recommend checking out The Honest Broker by .

3

This figure includes income earned from other jobs outside of music. If we summed up the sources of revenue from 2018 listed in the figure above, we get $16,882, which is just under half of the $35,872 that musicians reported earning in the Census (in nominal terms) on average that year.

It’s also worth noting that all of the above figures are means. Given how much top artists earn, the average skews a little higher than what the typical (median) musician might earn. Median earned income of musicians was only $36,000 in 2022, over $7k less than the average.

Still, the that gap between average and median is narrowing ever so slightly. This could be reflective of the challenges of making money from digital recording/steaming vs. analog formats. Less revenue from recorded music, and the rise of micro-culture over mass culture could be hitting well-known higher-earning artists more vs. local acts, who have benefited from people generally having more time and money to enjoy live shows.

The huge decline in agriculture and production jobs, and the emergence of STEM work from almost nothing, all technology-driven, is immediately visible. But aside from these major changes, this is quite a noisy chart. How can it tell us how normal or unprecedented the evolution of jobs in the recent past has been?

This second figure from the original Fed analysis, which measures how much the distribution of jobs in each decade varied from the previous one, helps answer that question. But the results surprised me. How could it be that jobs changed less in the 2010s than any other previous decade?

There were certainly some flaws and oversimplifications in the methodology. For example, it only measures changes across high-level categories. While it might capture major shifts like workers switching from farms to factories, or from factories to driving for Uber, it doesn’t count shifts between more similar lines of work, like going from data entry to customer service work.

A few years ago, I took a first crack at measuring the changes at this more detailed level, and visualized what they looked like in each decade up to 2019. It roughly confirmed what the Fed also concluded: not as many people switching occupations as back in the day.

But I was still unsatisfied. The definitions of jobs have shifted over time, and this is poorly accounted for in the original analysis. There were some significant realignments in 1970, 1980, and 2000. Could these make the rates of change observed in the past seem greater than they actually were?

After discovering tables like this from the Census, I found a way to largely control for this factor too.1 And the story remains similar, whether using Census (individually-reported) or BLS (employer-reported) data2. That is, until 2020. Let’s zoom in on the past three plus decades since 1990, and up to 2023.

Unsurprisingly, the pandemic and its aftermath changed a lot. The 2010-2020 change of roughly 16% of the workforce turning over is the most observed since at least 2003-2013. But rather than people returning to the same occupations after the pandemic, change continued to be above average through 2023, bringing the 10-year rate of change past its previous high in the 1990s!

It’s important to note that no single year was more significant than 2020, which saw about as much change as we typically see over 3-5 years3. But 2021, 2022, and 2023 were all above average individually, which continues to bring that 10-year moving average up. Even 2023 saw more than twice as much change as 2015, which was only a little less than where we were at in the depths of the Great Recession.

Comparing some of these key individual years, we can see how strikingly different they are. Even though there was a significant decline in clerical jobs in both 2015 and 2023, these losses have accelerated, and many other types of jobs can also be seen on the extreme ends of either faster growth or decline.

While the overall rate of change in 2009 was similar to 2023, there is a much clearer breakout by sector with big recession-era declines in blue-collar construction, production, and transportation jobs, and relative increases in healthcare and education. It’s also important to note that these figures are relative to employment overall, so even the “average” occupations toward the middle still saw significant losses in 2009. The reverse was true for the growing labor force of 2023.

The shift in work during the 2020s appears to be a lot more complex than what happened during the Great Recession. There is a lot more that we could dive into here, but let’s zoom back out and think of how this overall rate of change compares to history.

The big picture

We can make reasonable comparisons to more historical decades by using a broader standard for the definitions of each occupation that holds up better over time. Rather than measure the total amount of change across 406 occupations that are tracked from 2000 onward (and for Census data, backdated to 1990), we can use 246 more general occupations that still cover the full range of employment and backdate to 1970 while still controlling for the smaller re-definitions that have occurred since then.

Similar to the Fed analysis, we see even more change in the 1970s and 80s. Naturally, because we are tracking against the 246 broader occupations, there is less variance overall than if we measured the change across the 406 narrower occupations after 1990-2000.

But we can use an even broader standard of 133 more general occupations that go all the way back to 18704:

From this, we see that even the 1970s and 80s are at the back end of a wave of significant changes in the types of jobs that Americans worked. The 2000-2010s were unusually stable decades in terms of changes in the occupational mix, and the economy of the 1910s-1960s was about twice as dynamic in this regard!

Comparing this analysis with the original Fed one, we can see that in most years, changes in occupations within categories (e.g. from office clerks to secretaries) are usually much more common than changes across categories (e.g. farmers to truck drivers)— this difference is represented by the gap between the black line and the light blue one above vs. the difference between the black line and zero.

Excepting the Great Depression, 1910 through 1970 was an economically dynamic period. Meanwhile, the de-industrialization of the 1970s and 80s happened amidst a decline in the rate of occupational change, and the economic recovery of the 2010s happened at record-low rates of occupational change. There doesn’t seem to be a clear relationship between changes in the types of jobs and economic growth. What else are we missing?

Deeper questions

So far, our metric for occupational change has just measured how different a current distribution of jobs is compared to 10 years before. This is a good high-level overview that compares nicely with metrics used by the Fed analysis and the BLS5, but it misses a few things. For example, it doesn’t capture how much the changes were concentrated in a few specific jobs vs. spread out across the wider economy. This is a really important consideration: some of the steepest declines in employment in particular trades in the past have been technology-driven. As we anticipate how big of an impact AI and robots might have on our future job market, it can be helpful to see at a high level how often these sharp declines have occurred in the past.

Using a weighted standard deviation of job growth rates can help us see this, as the extremes factor more heavily. To ensure that significantly declining occupations are considered similarly to significantly growing ones, we should use annualized growth rates rather than 10-year ones6.

Tracking this shows that the 1910s and 1950s saw more extremes than the other years between 1910 and 1970. Both of these decades were driven by a notable exodus from agriculture. In the 1910s, it was farm laborers, but the 1950s saw an even sharper decrease in both laborers and landowning farmers.

Subtracting the average overall amount of change from the standard deviation can get us a measure that is similar to kurtosis and helps us highlight some other decades that saw unusually sharp changes relative to the overall average.

The 1910s and 1950s are still notable here, but it is the 1900s which really stand out in this case. When we look at the distribution of change in the 1900s, we see a relative decline in the percentage of the workforce who were landowning (or tenant) farmers. But in terms of absolute numbers, there were still more farmers in 1910 than there were in 1900! Employment in this area didn’t actually start declining until the 1920s (with a decline in farm workers preceding it in the 1910s).

This is because the wave of immigration and urbanization in this decade made growth in most other occupations dwarf the growth in farming.

So this historic example of a (relative) decline in farming isn’t something that should worry us too much in anticipating scenarios of job-replacing robots. But there is another decade highlighted by this metric that warrants more of our attention: the 1980s.

A key indicator of robots taking our jobs

The significant loss of manufacturing jobs was one of the defining features of the 1980s job market. While there was certainly growth in other occupations that offset this to some degree, increases in employment elsewhere were more tepid. This is quite a different scenario from all other decades from 1910 to 1980 where strong growth is some sectors was able to provide work for those leaving agriculture or other manual labor occupations.

There is one more metric which we can use to estimate the degree to which there was a balance between people being “pulled” into growing occupations vs. “pushed” away from declining ones: the weighted skewness7.

A positive metric here means the distribution is more heavily weighted towards quickly growing occupations; a negative one means it is more weighted toward quickly declining ones. The 1980s immediately jump out as the most negative decade by a long shot.

This chart tells an incredible story though: the labor market does tend to “push” workers from existing jobs rather than “pulling” them into new ones overall, but in some decades, the effect is more pronounced than others. Only in a few decades was growth in some occupations more concentrated in declines elsewhere. The 2020s labor market is one of those rare exceptions, depending on how you measure it8.

The implications of occupational change on technological development and politics

I think the experience of de-industrialization and its associated job loss in the 1980s and early 90s scarred us in America, and still has a deep-seated influence on our political rhetoric and priorities in the 2020s. Even with unemployment near record lows, and indicators like the ones above suggesting that the job market is being more driven by growing new fields rather than jobs being automated or outsourced away from declining ones, there is still a pessimistic outlook that colors the American perception of AI. These sentiments are different in other parts of the world that had a very different experience with technological advancements and occupational change over the past few decades:

Mentions in the literature of “job loss”, job creation, and related terms as measured by Google’s ngram viewer provide another interesting way to measure just how much of an impact the experience of job losses in the 80s had on public discourse in the English-speaking world.

Until the 2010s, mentions of these terms rose and fell with mentions of robots.

These fears are still very much in the public consciousness. And this makes sense. The median voter in the US came of age around 1990, and a second wave of job loss in manufacturing (and also construction) during the Great Recession was a near-repeat of this experience that many in their thirties and forties remember too.

What the indicators say now

New developments in AI and robotics, among many other factors, could accelerate layoffs again in the future. But when would we know if this actually happening?

While the above metrics of occupational change are based on Census ACS and BLS OEWS data that are some of the least timely indicators (coming out almost a year or more after initial surveys), there are ways to catch these changes much earlier. An unexpected rise in weekly unemployment claims from states might direct more attention to the next monthly Current Employment Statistics (CES) and Current Population Survey (CPS) reports from employers and individuals respectively. Further confirmation of any observations from the monthly reports could be confirmed about about a month later when results from the Job Openings and Labor Turnover Survey (JOLTS) are released. And if fewer job opening are recorded, the ratio of unemployed per job opening from this report can sometimes be a leading indicator of changes to come.

As of this writing, JOLTS still indicates that there are more job openings than unemployed workers, but the slight upward trend in initial unemployment claims means this ratio might rise in an unfavorable direction when April 2024 data is released in the first week of June. If this trend continues, I will plan to do a special analysis into what this change in employment looks like, and if it shows any similarities to previous technology-driven layoffs.

Where the numbers disagree

As I have taken a closer look at the long-term trends and puzzled over how they connect to the latest numbers, I’ve encountered a few challenges in reconciling differences between the various surveys. This probably merits some more in-depth explanation in a later piece, but a few observations for now:

• While the more comprehensive annual surveys break down employment by detailed occupation (as well as detailed industry), the CES survey only reports by detailed industry, and JOLTS only reports a breakout of high-level industry.

  • This is fine for occupations are so concentrated in particular industries (e.g. software engineers in Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services) that a significant change in industry employment can also suggest a change in demand for a particular skillset.

  • However, many jobs that might be under threat of being automated by AI (e.g. customer service representatives) are so spread out across various industries that it is hard to follow their trajectory month-to-month.9

• The more detailed and timely BLS employer-reported surveys like the CES don’t capture some types of self-employed and “gig” workers that are counted by Census surveys like the ACS and CPS. However, these can often be some of the fastest-growing jobs.10

With all this in mind, I hope to model out what the employment numbers might look like during a realistic rollout of self-driving cars, humanoid robots, or more autonomous virtual agents. Better understanding the rates of historic changes, and the context around them (including the ambiguities that are likely to arise) I think are critical for making projections that capture the complexities of the real world and aren’t overly reductive. One of the biggest questions that the 2020s and 2030s may put to the test is if the rate of automation might again exceed the economy’s capacity for adding new jobs. In their report from the spring of 2023, researchers from Goldman Sachs put this at about 0.5% per year, based on this paper from economists Daron Acemoglu and Paul Restrepo.

If technological displacement begins to exceed 0.5% of the workforce per year, will that intensify efforts to bring about a shorter workweek, or cause a “race to the bottom” for wages? Or have we entered a new phase of economic development where new jobs are being created fast enough to offset any displacements that might occur?

Predicting the new jobs of the future

I often wonder how much predicting the jobs of today would have been possible in the past. I made a chart similar to the original Fed one, but de-stacked it to more clearly see the trends in each occupational category.

One trend that seems quite apparent to me is the growth of the “Management, business & financial” category. As technology becomes more powerful, does this mean that managers are more empowered to do work themselves?

It took me many months to clean up the data behind this chart enough to be more comparable back to 1870.11 The depth and breadth of it go far beyond the high-level CPS occupational figures available in FRED— being able to easily drill down to each occupation individually can allow one can see details of professions that declined dramatically as examples of how future technological displacement might play out. The complexity of the significant changes in occupations in 2023 wouldn’t be visible any other way. There are so many stories behind this data that I look forward to sharing over the coming months!

1

The way in which occupations have been reclassified over time is quite confusing, so it’s best to give a specific example with a visual:

“Insurance and other financial products agents and brokers” is an occupation that has been tracked consistently over time, but some workers classified as “General office and administrative support workers” in most other years have occasionally been included as well. A Sankey diagram can be one of the best ways to visualize the impact of these changes in definition:

As we can see, the definition of this occupation in 1970 is much more broadly inclusive, but only temporarily. The growth in employment between 1960 and 1970 is almost entirely due to the change of definition, which is adjusted again in 1980 to exclude many of the workers who were counted in 1970 but not in 1960. To reduce this error, my estimates of employment in occupations in 1970 are based on the Census tables referred to here which estimate how many workers in 1970 would have been in each 1980 occupation, and how many workers in 1960 would have been in each 1970 occupation. Carrying the 1960 ratios forward and comparing them with the 1970 employment estimates based on 1980 occupations ends up matching up nicely in this case.

Note that the jump in the 3rd stage just shows the increase in employment in this field between 1980 and 1990 when no change in definition occurred.

Also note that these aren’t the actual Census classifications, but a “max comparability” classification that I developed specifically to reduce the amount of error occurring across more narrowly defined Census occupations (which I have combined in many cases where the evidence of these shifts in definition have supported it— I address this in more detail in another footnote below).

4

The 2015 Fed analysis of occupational change cited above uses 1950 occupational categories (OCC1950). However, some of these completely disappear after 1970 or afterwards due to the reclassifications of 1970 and 1980.

To minimize these losses over long time horizons, I created three series of inter-related occupational classification systems:

• A child “Post-1990” series (explained in this footnote above).

• A parent “1970 broad” series

• A grandparent “Max comparability” series

The parent and grandparent “1970 broad” and “Max comparability” series combine some of their child occupations such that there is either a 1:1 or 1:many relationship between parent and child. This allows for the lineage of newer “child” occupations to be more clearly defined. Looking at a child series allows for a more detailed analysis from either 1970 or 1990 going forward, while looking at a parent series allows for a rough comparison of broader occupations over a longer period of time in a way that can be adjusted to be consistent across the 1970, 1980 and 2000 reclassifications. The broadest “Max comparability” series allows for rough comparability of occupations in Census data going all the way back to 1870.

While the Post-1990 series is roughly based on the OCC2010 occupations from IPUMS, the “Max comparability” series is based on their OCC1950 occupations, with similar caveats for pre-1970 data:

• Many OCC1950 occupations, in particular ones that later disappeared, are combined into more general occupations. And unlike the relationship between my “Post-1990” categories and OCC2010, no “Max comparability” occupations are defined more narrowly than the OCC1950(s) they are associated with.

• There are also a few occupations whose OCC1950 designations in 1970 I disagreed with and reassigned to a different OCC1950, but there are only 3 of these and all of them were very small.

Between using this table to make estimates of employment in 1970 by OCC1950 consistent with how occupations were defined in 1960 (assuming the proportion of reassigned workers remained the same in 1970), and using this table to bring accurate OCC2010 estimates of employment back to 1990, getting an accurate count of employment from 1870-1970 and from 1990-2022 was relatively straightforward. Building the intermediate “1970 broad” series that bridges these two systems such that there is only one parent for each child occupation was the most difficult task.

There is an intermediate OCC1990 series available that some researchers use to bridge this gap. However, this series has its limitations:

• Some OCC1990s effectively have multiple OCC1950 parents, which makes comparability across 1970 challenging unless some of the individual OCCs under the OCC1990 are changed, or the OCC1950 is broadened into a “Max comparability” category (I have taken both approaches).

• Some OCC1990s do actually offer more detail than the OCC2010 occupations that succeed them (particularly production ones). However, if we want to have a series that we can reliably compare from 1970 going forward, the intermediate “1970 broad” series must be at least as broad as the “Post-1990” series (which itself is mostly based on the separate OCC2010 series).

The complexity of the adjustments I made to achieve these three more comparable series are hard to explain in words alone. Here are a few Sankey diagrams that explain the problem for one of the occupations that was most difficult to track consistently over time, and my solution:

Note that in the following charts, the dates refer to the original OCC years; 1960 and 1970 figures represent estimates of 1970 employment across the two different systems, and 1980 and 2000 figures represent similar estimates of 1990 employment.

Here are the original Census occupations (OCCs) for each year 1960-2000 and transfers between them for all workers counted in my max comparability occupation “Retail and other sales workers”

What a mess! Here’s how these occupations look when put into their broader OCC1950, OCC1990, and OCC2010 categories:

Noticeably cleaner, but there is still a lot of inconsistency. By reassigning a few of the child OCCs to OCC1990s that are a better fit, we can further reduce that error across years:

However, my proposed system of “Max comparability”, “1970 broad”, and “Post-1990” is even cleaner: we can see a clear generational progression here from a general “Retail and other sales workers” category with some specified sales representatives broken out in 1970, and other more detailed sales occupations broken out in 1990-2000.

Some error still exists, but it is small enough to control for it in my other charts in two different ways.

The first approach is to show values for both the previous and new definitions of an occupation for each year that a redefinition occurs:

While this chart doesn’t show the details of what other occupations some individuals were reassigned from and to in the key years, it does show conspicuous drops that make it clear that a reclassification happened. At the same time, these drops are small enough that we could conceivably imagine how many people would today be counted in the earlier definition of an occupation by assuming* that the (small) difference has grown proportionally with overall employment.

*Note that this is an assumption we have to be most careful about in occupations that have seen significant decline, somewhat careful in occupations that have seen significant growth, but can tolerate more error if the occupation in question has grown roughly in proportion with employment.

If we want to make this assumption and imagine how many workers today might have been counted in the pre-1970 bundle of old occupations represented by the “Max comparability” occupation, we get a simple chart like this:

Designing these three series took a considerable amount of time and judgement, but the resulting occupations are significantly more comparable over time than the existing OCC1950, OCC1990, and OCC2010 designations. Here is how much error was reduced using max comparability series across 1960-70, and using 1970 broad series across 1970-80 and 1990-2000 relative to the OCC1990 occupations:

Across each of these key years, between 16 and 26% of workers had their occupations reclassified by the Census in a significant enough way to shift them into another OCC1990 occupation. I was able to slightly reduce this by changing which OCC1990 70 different individual Census occupation codes were assigned to, and more significantly reduce the error by combining multiple OCC1990 occupations into broader “Max comparability” and “1970 broad” ones. These approached occasionally reintroduced some error (mostly from there being a minority of workers who were correctly designated in their original OCC1990, but outweighed by a majority who were not). Nevertheless, total error was reduced by nearly 80% across the 1960-70 transition, and by roughly 50% across 1970-80 and 1990-2000.

When considering these broader definitions for some occupations, there are both inflows and outflows that roughly balance each other out. For example, the 1970 broad occupation “Medical assistants and other healthcare support occupations, nec” sees many workers who had some practitioner duties moved into other practitioner-specific healthcare occupations (e.g. “Licensed practical and vocational nurses”), but this outflow is roughly balanced out by some clerical and technical workers such as “Cashiers” and “Other health technologists and technicians” moving into the new 2000 classification.

If we want to track salaries and hours, a 16% outflow offset by a 20% inflow is still a significant change that one should be cautious about making assumptions about. But if we are just adjusting for the overall number of workers, a 4% change is easy to control for. Overall, the distribution of net transfer error across 1960-1980 (combining the 1960-70 and 70-80 classification changes) is under 20% for most max comparability occupations.

The reclassification of 1990-2000 cut some occupations a little deeper, with nearly 25% of workers in occupations whose definitions shifted by 20% or more on net. However, there is less time to account for after 1990, so more error can be tolerated. The occupations that I allowed to remain distinct even with a larger amount of transfer error tended to be ones where employment growth was roughly in line with overall employment growth, with a few exceptions. However, these designations may need to be revisited should employment shift significantly in the future. A better approach at that point may be to use AI to analyze how many historic occupations would have fit into a more modern classification rather than trying to imagine how many people in today’s workforce would have fit into 1950, 1970, or 1990 occupational designations.

5

It’s worth noting that the BLS metric for occupational change is slightly different from the approach used by the Fed and my own analysis. The formula I use is as follows:

Essentially, I take the total difference between the initial and final year employment for each occupation, and divide it by total final year employment. The Fed analysis takes this approach as well, but only measures major category differences rather than individual occupation changes.

However, the BLS effectively takes how much each occupation has changed as a percentage of overall employment, and sums that up to a total that ends up being numerically similar to my formula for calculating the total amount of occupational change, but the units are not technically expressible since the denominators are slightly different:

It’s also worth noting that my approach includes all occupations, but only at the broader “Post-1990” level. I make some interpolated estimates (detailed in this footnote above) where changes in the definition of one of these broader occupations likely occurred. Meanwhile, the BLS measure of occupational change is calculated using the more detailed SOCs, but excludes SOCs that merged together in later years or were otherwise transformed such that the changes have a many-to-many relationship.

Effectively, the broadness of my post-1990 classification system reduces the overall occupational change estimate a bit. But the exclusions reduce the BLS’ overall estimate as well, resulting in similar numbers of 18.9% and 17.8 for 2012-2022 (the “sum of absolute differences” for the Naïve model effectively measures the total change in occupations between 2012 and 2022).

7

ChatGPT really helped me learn how to measure weighted skewness. Knowing that it has a tendency to sometimes make things up, I spent a bit more time validating what it told me than I would if I read the same content in a textbook, but I think this exercise helped me learn the concept even better than I might have otherwise. Here is it’s explanation, which checks out as far as I am aware:

Calculating the weighted skewness of a distribution in Google Sheets requires a bit more effort than calculating the basic skewness, as Google Sheets does not provide a direct function for weighted skewness. However, you can use a combination of formulas and steps to achieve this. Weighted skewness takes into account the frequency or weight of each value in the dataset, providing a measure that's more representative of the distribution when each observation doesn't contribute equally.

Here's how you can calculate the weighted skewness:

Step 1: Prepare Your Data

Assume your data is in two columns: Values (A) and Weights (B), starting from row 2 to n.

Step 2: Calculate Necessary Statistics

1. Weighted Mean:

   • Formula:

     

   • In Google Sheets, if your data is in A2:A, and weights are in B2:B, you can calculate the weighted mean in a cell with:

   =SUMPRODUCT(A2:A, B2:B) / SUM(B2:B)

2. Weighted Variance:

   • For each value, calculate the squared difference from the weighted mean, multiply by its weight, sum all these products, and divide by the sum of weights.

   • Formula:

     

   • This needs to be done with a combination of SUMPRODUCT for the numerator and SUM for the denominator.

3. Weighted Skewness Formula:

   • Skewness is a bit more complex to calculate directly in Google Sheets due to its formula:

     

   • You'll calculate the numerator as a sumproduct of the weights and the cubed differences from the mean, and the denominator as the sum of the weights times the cubed root of the weighted variance.

Step 3: Implementing Weighted Skewness

Due to Google Sheets' limitations, you'll need to use helper columns to calculate the squared and cubed differences from the weighted mean, multiplied by their weights, for the variance and skewness calculations, respectively.

Note:

Directly implementing these calculations in Google Sheets, especially the weighted skewness part, will require creating and managing several intermediate calculations and potentially large helper columns, which can be cumbersome for very large datasets.

If your dataset is sizable or you require more sophisticated statistical analysis frequently, consider using statistical software or programming languages like R or Python, which offer more direct and powerful functions for such computations.

8

This metric is more positively skewed in the Census moving average (MA) due to some shifts in occupational definitions in 2018 that my methodology hasn’t fully accounted for. When measured annually, weighted skewness was also useful for identifying other years that had similar shifts and controlling for them:

Other than in 2018 though, we see broad agreement across Census and BLS data that the strongest changes are coming from growing occupations rather than declining ones, pushing the skewness into new positive territory. It should be no surprise here that the Great Resignation of 2022, in which many workers left their old jobs for greener pastures elsewhere, was one of the most positive years on record for growing occupations driving more change than declining ones.

9

The smaller sample sizes of the Current Population Survey mean that month-to-month data even for larger occupations like Customer Service Representatives can be quite noisy:

It’s not unusual for employment to move by 2-3% in a month, which would be significant if sustained, but hard to determine a trend from one month of data alone. The larger occupational categories tracked by FRED might be broad enough to avoid this sort of noise, but the number of smaller occupations contained in the broader ones adds noise of its own.

(Thanks again GPT-4o for the quick and easy chart!)

10

The growth of “Couriers and messengers” in Census data lines up quite well with interest in Doordash. But BLS data actually shows this occupation as in decline! This is likely due to its surveying of established businesses rather than independent contractors like Doordash delivery workers. This chart captures the strengths and weaknesses of both datasets: BLS data is more timely, but Census data is often more comprehensive (though workers in some areas may not identify their occupations as precisely as their employer would):

11

A note that this high-level chart is based on the “Max comparability” categories I detailed in the above footnote here. A further point of clarification on these: while every “Post-1990” occupation has no more than one “1970 broad” parent and one “Max comparability” grandparent, the relationship between each of these occupations and higher level designations that group multiple them together into larger categories is different.

Similar to the BLS Standard Occupational Classification (SOC), there are three levels of higher-level designations that occupations can fit under. From broadest to narrowest, they are:

• Functional major category

• Functional minor category

• Functional broad occupation

Unlike the BLS SOC, not every occupation is assigned to a “Functional broad occupation”. Sometimes, the “Functional minor category” goes deep enough such that a third level category would be superfluous and just repeat the name of the occupation.

I also do add the “functional” qualifier to each of these categories because while they sometimes line up with the BLS designation, I have often made some changes, including:

• Shifting BLS categorizations down a level to have fewer top-level categories

• Shifting BLS categorizations up a level to highlight a particular category as more distinct

• Allowing for supervisory and managerial occupations to be flexibly categorized as either part of the “Management, business & financial” major category, or as part of the category of worker who they are in charge of supervising

• Creating a new category that I think adds value for comparative analysis

• Shifting occupations to a category that is different from how it is categorized in BLS’ system for some other reason

• Minor name changes

Aside from these differences from BLS categories, another key difference to be aware of is that the “1970 broad” and “Post-1990” children of a “Max comparability” occupation may belong to a completely different category from their parent. It is often the case that as occupations became more specifically defined over time, these realignments were necessary.

For example, the “Supervisors, managers, and executives” max comparability occupation belongs to the “Management, business & financial” major category. But “Supervisors of transportation and material moving workers” is split out from this as its own “1970 broad” occupation after 1970, and going forward, this occupation can be flexibly categorized as part of the “Transportation & moving” major category. This is the case for many management occupations, which is why looking at the same chart on an unadjusted basis, which looks how each level of occupation (Max comparability, 1970 broad, and Post-1990) side-by-side and without any transfer adjustments will show such big jumps in the key years of 1970 and 1990.

However, looking at the same chart strictly on a max comparability basis, even with no adjustments for transfer error (explained further below in the above footnote) doesn’t show as significant of changes. This is because on this basis, 1970 broad and later occupations such as “Supervisors of transportation and material moving workers” are still considered to be a part of “Supervisors, managers, and executives” and the higher level categories it belongs to:

Here are some of the recent developments that have stuck with me the most:

• AI agents taking on increasing responsibilities: from handling up to 2/3 of customer service inquiries in some cases, to doing highly technical work such as writing and deploying (simple) web applications, human-level data analysis tasks, and aiding in the discovery of whole new classes of antibiotics.

• Early signs of generalist humanoid robots that can understand voice commands and do household tasks. Will they soon be able to (economically) do what these cleaning robots cannot yet? (It should be noted that the time it took for the robot to do this was 8-10x longer than a human, so the trajectory of improvement here will be something to watch)

• Accelerating investment: NVIDIA stock has doubled since my previous exploration of AI trends only 5 months ago, becoming the 3rd most valuable company in the world by market cap while branching out into robotics as well.

On the other hand, critical flaws have become increasingly clear, with no easy fixes in sight:

• Looseness with facts: from generating false legal citations to offering customers a Chevy Tahoe for $1 when prompted the right way, the internet is full of (often funny) stories of AI fails.

• Impressive benchmark results ≠ real-world impact: many common AI benchmarks that make systems seem really smart effectively just measure the ability to memorize information, a domain where computers have long vastly exceeded human ability.

• Less automation than advertised: seemingly autonomous systems like robotaxis still rely on an appreciable amount of human intervention.

In spite of these challenges, the LLM-fueled advances of the early 2020s appear to open a lot of doors for AI agents and eventually robots to take on a lot of work. But with U.S. unemployment still near record lows and layoffs still well below their historical average in March of 2024, it’s clear that robots haven’t been throwing many people out of work yet. However, productivity is on the rise again in spite of only 5.5% of firms reporting use of AI to the Census. Given that earlier surveys have suggested AI use by individuals may be higher, it’s quite possible that there is a substantial amount of “secret” AI use by employees is that is creating efficiencies that aren’t yet captured in productivity statistics.

Past performance is no guarantee of future results of course. While economists who have extensively studied the history of the labor market like Daron Acemoglu think AI and computer vision abilities will automate less than 10% of tasks in the economy, studies like this one from Goldman Sachs in April 2023 estimate the level of automation as closer to 30%. Some technologists are even more aggressive in predicting the arrival of an Artificial General Intelligence (AGI) so smart that it exceeds human ability in all areas by 2030. But will speculation about AGI being just around the corner seem just as foolish as those who predicted fully self-driving cars by 2019?

One thing that has bothered me about many predictions of AGI is how much they tend to oversimplify intelligence as a single variable, even as the definition of what AGI means is hotly contested. Yes, it is hard to speculate about the specifics of something that doesn’t exist yet, and whether scaling up raw computing power alone will solve problems is an area of active debate. Papers like this one from NBER do account for the existence of tasks of greatly varying complexity and the likelihood of some jobs staying “nostalgically” or legally human. And even my own previous analysis, which looked at the impact of the most transformative AI I could imagine from the perspective of working hours, was far simpler than I would have liked.

A few years ago, I did a deep dive in Census occupational data, building off this Fed analysis that looks at how much jobs have changed in each decade going all the way back to 1860.

I would love to see a chart that projects this distribution of jobs forward to the year 2120. But even a 20-40 year prediction could capture many of the hypothesized impacts of current and future generations of AI. McKinsey has published a good snapshot of what the current state of occupational change looks like, and some rough estimates extending further into the future (see figure 8 here). My goal over the coming months is to play around with a few different scenarios in a way that more ordinary readers (especially students who are making big decisions about their careers) can appreciate.

This is a story with many parts to it— I will keep these links updated as I continue to explore this subject:

• At a high level, how can we measure how much jobs are changing? How do the 2020s compare to history? And how can we best communicate this to the public?

• Is a robot or a recession more likely to take my job? How does technological change compare to other factors?

• If robots were about to take our jobs, where would we see the first signs where that was about to happen?

• Where have previous official predictions of job change failed?

• How predictable is the impact of technology on jobs?

• Which jobs should we be most worried about in the 2020s? Which ones might grow even more than expected and still exist for a long time to come? And which ones should we keep the closest eye on?

• What are the implications of self-driving technology on the job market?

• How might AI reshape education and job training?

• Are jobs becoming more or less satisfying over time?

• What is the role of unions in shaping the job market?

• Taking likely job changes into account, what does the future of the workweek look like?

The beauty of these topics is that they are all relevant questions for the full range of reasonable scenarios of technological advancement. In thinking about all of these various facets of the future of work, I am struck by how complex and specialized each occupation is. Context is that which is scarce, as puts it. The AIs of the future will have a lot to learn, as will we in figuring out how much we can trust them.

Writing this piece has been a exercise in helping me better contextualize breaking news from the field, and think about what the impact of today’s technology might be on the far future. After reading, hopefully you’ll have a good enough level of interactional expertise to be able to make sense of the excellent work of the many writers and researchers I’ve cited here, and potentially get ready do some work in the field yourself.

About my own background: I have a B.A. in economics and 10+ years in public service, mostly in technology-related operations. I am not a specialist in AI, but rather someone who is generally interested in making sense of various predictions about the future. In my past 2 years of writing this newsletter, I’ve done a deep dive into how we got to a 40 hour workweek and where we go from here, including a base case and an aggressive prediction about what that might look like in the year 2120. I’ve also explored how the nature of work has changed over time, and plan to do some more specific predictions of this as well. Some key questions I’ll be asking myself include how transformative AI and robotics will be compared to the impact of mechanized agriculture, and if employment in any other fields might be curiously stable in spite of new technology, like construction has been since 1910.

Substack estimates this is about a 70-minute read, but there are some high level summaries at the top for you to go directly to the parts you might be most interested in. Over the course of this piece, I’ve referenced over 300 different articles from over 100 unique sources, so if you read my summary and dive into some of the linked pieces, you should give yourself credit for reading a book!

Without further ado, here are the 6 key AI variables to watch:

1. Computing power and price: the scale of this is already massive and steadily growing as per Moore’s Law, but there is only one more order of magnitude of conventional growth left before hitting some hard physical limits in the 2030s. Price-performance gains have slowed and might slow further as these limits are approached.

2. Model size: the size of AI models grew rapidly around 2020, but could hit some limits (at least for high-quality text) as early as 2024. Image, video, and other models may continue to grow into the 30s and 40s, but inference costs, slower progress in memory bandwidth, and difficulty scaling the allowable size of prompts are important constraints that might limit growth to just two orders of magnitude over the next decade.

3. Accuracy and alignment: AI advances past human ability in many areas, but has appeared to level out below human ability in others. There are many benchmarks for evaluating performance, and just as many debates about the validity of those benchmarks. The challenges of measuring the safety of self-driving cars is an instructive example, and there is a lot we can learn from prediction markets on how we can better visualize progress and mitigate potential harms.

4. Algorithmic efficiency: like the quality of software in general, this is hard to measure. However, we’ve seen significant advances in many areas here, including making more efficient use of data and parameters, better techniques for fine-tuning, the development of interpretability features, interactions across different modes (e.g. text vs. image), and human skill at using AI. Whether the improvements here are easily reproducible in the open source world or not will have significant impacts on how AI develops going forward.

5. Investment: the scale of money invested in both AI hardware and software has been increasing rapidly, eclipsing spending on robotics, though there is a wide range of uncertainty in these predictions. As of October 2023, the stock market appears to have priced in 2% more annual growth than usual over the next 10 years.

6. Public opinion and regulation: these aspects touch all other variables in some way, and accountability, ownership, and agency are important concepts in law. Where automated decision-making could be highly consequential, regulation is likely. Public opinion and trust in AI tends to be lower in Western countries and higher elsewhere with some exceptions and caveats, though worries about jobs and the economy are significant everywhere, and risks posed by AI are difficult to weigh against each other. To stay informed about trends related to regulation, privacy advocates have created comprehensive legislation trackers across all levels of government.

The bottom line: transformative AI that could automate a significant amount of work is possible within the next decade, but the reality is likely to be more complex. Better understanding accuracy and alignment is important, and what I intend to focus on next.

** Note that this is a living post. I intend to update it as significant breakthroughs happen. Where there’s uncertainty among experts, I’ve tried to explain as concisely as I can, but please comment if there’s anything you think I’ve gotten wrong.

Please do bookmark, subscribe, and share if you’ve found this guide useful!

1. Computing power and price

Computers, and the artificial intelligence they enable keep getting more powerful. From beating the best human chess players as early as the 1990s to embedding themselves into countless aspects of daily life in the the 2010s to passing business school exams and Amazon coding interview questions in the 2020s, it’s left many wondering when machines might surpass most or all of our human abilities. Some forecasters have based their predictions on a mechanistic comparison between the computing power of computers and that of the human brain (often referred to as “biological anchors”). But what do the large numbers of FLOP/s in predictions like these mean?

How to measure and predict the power of computers?

The power of a computer is typically measured in the number of floating point operations per second (FLOP/s) it can perform. For example:

• The best individual computers (based on GPU performance1) in 2023 perform in the teraFLOP/s range (1e12-14, or trillions or FLOP) in 2023. These surpass the very best supercomputers from around the year 2000.

• Hundreds or thousands of individual GPUs can be parallelized together to make supercomputers in the petaFLOP/s range (1e15-17, or quadrillions of FLOP).

• Tens and hundreds of thousands of individual GPUs can be put together to make world’s current fastest supercomputers, in the ExaFLOP/s range (1e18+, or quintillions of FLOP).

So, will our individual computers one day be as powerful as today’s supercomputers? According to Moore’s Law, which has held relatively steadily since the 1960s, if computing power doubles every two years (or 10x every 6½ years), we could get there between 2049 and 2062. However, there are a few other factors to consider.

The limits to Moore’s Law

On a price-performance basis, progress since ~2013 has been slightly slower than what Moore’s Law would predict. While the number of transistors continues to increase, the price per transistor has stagnated, so price-performance improvements are limited to other factors, like the number of logical cores.

On the other hand, the computing power of the largest supercomputers continues to grow slightly faster than Moore’s law, growing by an order of magnitude in just 5 years thanks to increased investment.

However, this rate of growth could also change as we approach some possible absolute limits. Sometime in the 2030s, the size of transistors may approach the size of individual atoms.

This means that we might only ever see individual processors that are only about 10 times as powerful as today’s. But if AI models trained on supercomputers continue to have many practical applications, and scaling them up continues to provide value, we could see ever more powerful and expensive machines, thousands of times faster2 than current ones. And if we see more practical breakthroughs in quantum (which some are skeptical of) or neuromorphic computing, entirely new paradigms3 will be opened up.

But what is done with all of this computing power?

2. Model size

If the 2010s were the decade of big data, the 2020s have been the decade where enormous amounts of data have been fed into the neural networks that power Large Language Models (LLMs) like ChatGPT, as well as others trained on images, video, and other modalities. The initial step of pre-training a model results in the creation of a foundation model, which can later be fined-tuned. While these tools can sometimes seem magical, at a high level, they are generally just predicting the next most logical word (or pixel, etc.) based on the statistical associations present in their data.

The large amount of data, and associations between each data point— in other words, the size of these machine learning models— is staggering: if you thought an exaFLOP was big, just imagine running an exaFLOP computer for about 10 million seconds (116 days). 1e18 FLOP/s times 1e7 seconds brings us up to 1e25 FLOP (minus some loss), or about half the size of GPT-4, the world’s largest LLM behind ChatGPT’s “premium” tier as of September 2023. That’s similar to the number of individual molecules in a quarter cup of water, which I think is a helpful way to think of the size of these models as they continue to get bigger4.

What makes AI models so big?

It’s helpful to think of the power required to train a foundation model as a function of two dimensions, tokens and parameters.

The number of tokens and parameters are relevant for determining the cost of two other fundamental aspects of training and using AI models: the forward pass, inference, and the context window. More detailed definitions of each of these terms are below:

Tokens

This is just the quantity of data that a model is trained on. For language models, there are about 1.3 tokens per word. The Common Crawl, which scrapes about 10% of the entire public internet, consisted of about 1.4 trillion tokens in 2019. GPT-4 is estimated to have been trained on about 10x this number, or about 100,000x the total number of words a child encounters by age 10.

Parameters

Each token has a number of weights associated with it to help it figure out what word it should output next. These weights are called parameters, and best practice as of 2023 (based on the results of the frequently-cited “Chinchilla paper”) suggests the use of one parameter per roughly 20 tokens.

Forward pass

Every time a model is queried, an operation known as a “forward pass” is performed to predict the next token that will be output as your response, based on the values of its parameters. During training, the parameters associated with each token are constantly being updated, making each forward pass very large. Furthermore, 6 forward passes are required during training to query all of the data in the model and perform the update (a process known as backpropagation), so the total “size” of the model is the number of tokens x the number of parameters x 6.

Inference and the context window

A simple query of the model after training has been completed is called inference, and it is much less computationally expensive than training. While it still is based on the number of parameters, it is only multiplied by the number of tokens in the “context window” (e.g. how many words are in the question you asked ChatGPT) times the number of attention layers (the “depth” of the model, which I explain more here). With 10 to 1,000 tokens per prompt, and roughly 100 layers, that’s just 1,000-100,000 operations instead of trillions. And since inference is only 1 pass rather than 6, querying a model is 8-10 orders of magnitude cheaper than training it— fractions of a penny compared to millions of dollars even for the largest models.

However, the weights associated with each token in the context window must be stored in memory. As context windows larger than 1,000 tokens are needed for queries that involve following through on complex tasks and remembering the types of requests we have made in the past, the practical size of these inputs when multiplied by an ever-larger number of parameters is constrained by the amount of memory available to do these inference operations. We will explore the implications of this in later sections.

The rapid rise and likely limitations of LLMs

13 trillion (1.3e13) tokens x 280 billion (2.8e11) parameters x 6 = 22 septillion (2.2e25) FLOP for GPT-4! What’s even more remarkable is that this is 100 times larger than the biggest model (AlphaStar) in 2019. Forget just doubling every two years! The recent trend has been almost 10x every two years 😲, thanks to increased investment. But how long can this trend continue?

By 2025, forecasters estimate5 that the largest training run might be between 3e25 and 1.2e26 FLOP (1.4x to 5.5x larger than GPT-4). By 2032, they estimate between 3.9e26 and 2.3e28 (~20x to ~1,200x larger). This is a bigger range of uncertainty than there is about processing power, because there are even more factors that may or may not limit the size of future training runs.

Training time and inference cost constraints

Further increases in model size mean that either computing power or the length of training runs will need to increase. But as mentioned earlier, conventional computing power per processor may only have one more order of magnitude of room to grow. And training runs longer than a few months are likely to be impractical. Increased investment seems like the most viable way to push through, as long as it is economically viable.

However, a bigger model also means inference is more costly. As of October 2023, it costs roughly 2 cents for GPT-4 to read a ~30 word question and give a ~100 word answer. A 10x bigger model would mean either a single query would cost 10x as much or take 10x as long, which is likely unsustainable given that ChatGPT queries already cost significantly more than Google Search ones. Decreases in computing power cost might make larger models more feasible, but other algorithmic improvements that maximize efficiency are primarily responsible for GPT-4’s faster-than-expected inference times.

Memory bandwidth

Memory bandwidth is the factor that keeps inference costs high for models past a certain size, and as mentioned above, is a significant constraint in developing models that are more personalized and can remember more about previous requests you’ve made. Improvements in memory bandwidth have also been significantly slower than increases in processing power. This constrains even what’s possible with increased investment.

To address this, recent efforts have focused on effectively reducing either the size of each FLOP (i.e. reducing the number of decimal places in calculations), or the overall number of FLOP needed to attain accurate results. When encountering estimates of FLOP/s, it’s helpful to know what the level of precision is6. Other algorithmic improvements to improve parallel processing of attention like FlashAttention might also compensate for this apparent bottleneck.

Limited data

There is also a limit to the amount of data available to use to train AI models. With current LLMs already approaching the size of the entire public internet, we may run out of high-quality text to train bigger ones as soon as 2024. Image and video data are still plentiful though and could continue to grow for another 20 years. This paper has an excellent illustration of the size and rate of growth of various modalities (text or speech vs. images or games).

Model size: the bottom line

Only the most aggressive predictions for 2025-20327 estimate that models will continue growing apace at their early 2020s trends. Given that improvements in processing power and memory bandwidth seem unlikely to accelerate, and the level of investment is already in the hundreds of millions of dollars for the largest models, I think the median forecasts of 2 orders of magnitude of further growth over the next decade (1 from hardware improvements and 1 from increased investment) are much more reasonable here. But even if growth in the size of the largest models cools off, there are many smaller models8 in development, and gains in their efficiency and effectiveness are scaling more quickly than anything else right now.

3. Accuracy and alignment

All of the computing power and data in the world is meaningless if we can’t use AI tools to do anything useful. While models at the forefront continue to improve according to various benchmarks, the degree of improvement varies significantly based on the task. Charts like the one below which track performance9 on various language and image recognition tasks make it seem as if AI’s abilities are either superhuman or rapidly accelerating toward that level. I explore these more below in my section about multimodality, but abilities like these, paired with how well they appear to be able to do creative work, are already affecting the livelihoods of writers and actors.

That said, there are other metrics where AI ability appears to level off below the ability of the average human, like visual commonsense reasoning10. For example, a state-of-the-art robot from July 2023 still struggled to understand the dynamics of certain objects, like how a round pen may roll when pushed, or the optimal way to push an object with an oblong shape like a banana. AI systems tend to struggle with other forms of reasoning as well, like understanding causality. While this piece from Qualcomm hypothesized causality being the “missing link” of the AI puzzle back in September of 2022, over one year later it appears that not much has changed, especially when you compare how much image-generation tools11 improved in the same time frame.

How to evaluate AI?

It’s clear that the simple Turing test proposed in the 1950s no longer captures the subtleties of where AI is improving most. Many of the examples cited above are tracked in Stanford University’s annual AI Index Report. This is an excellent source of information on many different AI-related trends, but this it is just one among many. Papers with Code shows progress across hundreds of different abilities in real-time, segmented by category, but it’s hard to know at a glance which benchmarks are most relevant12. Finally, LLMonitor provides an alternative way to view benchmark results— it will just show you what answers different models return to standard questions side-by side, albeit just for a few simple examples as of this writing.

While these benchmarks give us quite a bit of information about how AI’s abilities are improving, they also come with many problems. Questions from the test (or ones very similar to test questions) might show up in the training data, with the results potentially overstating an AI’s actual level of performance— this is referred to as “leakage” and it has created a significant reproducibility crisis. There is also the age-old problem of Goodhart’s Law— teaching to the test and then reading too much into the results. How can we get around these issues and ensure that benchmark results translate into meaningfully measured abilities to complete useful tasks in the real world? This is going to be a challenging issue to solve, but making progress here will allow us to make better predictions about AI’s trajectory.

One way of approaching evaluation is to think about some of the gaps in the current benchmarks. Measuring a model’s response to an immediate question based on the vast array of data it has been trained on is something that we might expect a computer to excel at. But this is a very different task from one that tests an AI’s ability to come up with multi-step plans and follow through on them in environments where new information comes up and constantly needs to be evaluated without catastrophic forgetting of the earlier context13. Admittedly, it might be hard for such a test to be standardized, but we will need to get creative. There is a lot to learn from the progress and pitfalls of other AI systems are already replacing significant amounts of human labor or coming close.

Example: measuring the safety of self-driving cars

Thanks to some more detailed analysis by , it seems like Waymo’s fully self-driving cars likely surpassed the safety record of human drivers, at least in the limited situations where they have been applied (3 million fully autonomous miles on the streets of San Francisco and Phoenix). If this is true, are self driving cars imminent? Why does this prediction market still have a wide range of uncertainty (between 2024 and 2030) about when they will be widely available?

The uncertainty comes from two sources:

• 3 million miles is still not a whole lot of distance when it comes to fully comparing the statistical safety of self-driving cars against the trillions of human-driven road miles each year in the US alone, so error bars on these initial reports are high.

• Public trust may be wavering and could be tricky to maintain as even statistically safer cars would still cause headline-grabbing accidents.

That expansions in service to several new cities were announced in August 2023 means that the next 3 million miles should come much more quickly than the first. And better transparency on several levels could help improve the public trust needed to continue piloting this technology:

• Cleaner independent visualization of government data about self-driving car accidents, contextualized alongside human accident rates.

• More willingness among self-driving car developers like Waymo to share more detailed internal data about how their vehicles handle exceptional situations.

• Tracking other externalities like those outlined in this report. To what degree might the cautious driving style of self-driving cars and other technical errors be responsible for slowing down traffic? How much of a risk does hacking pose?

What prediction markets tell us about other AI capabilities

Just as prediction markets can be helpful in visualizing when self-driving cars might be available, they can also help us understand progress in other transformative technologies, and how the latest news impacts their timelines. Of the many AI prediction markets out there, this one is my favorite because it assesses practical skills and not just exam results of a more “general” AI. As of writing this in September 2023, this market estimates a 25-75% confidence interval between 2026 and 2045, following the 2024-2030 window for self-driving cars. If the 2030s are as wild as this detailed forecast, how can we prepare ourselves for the tumultuous times ahead?

An idea for improving AI benchmarks and predictions

Reports by Goldman Sachs and McKinsey from the spring of 2023 take a look the impact of AI’s generative abilities on the labor market and economy at a fairly detailed level, allowing for some uncertainty in how capable generative aspects of AI will be. McKinsey has since followed up with a survey tracking AI use by industry and job function. However, some of the data behind their analyses are understandably private, and these only capture a moment in time in a period of substantial technological change. How can we make publicly-available forecasts more agile?

What if we had something that combined the granularity of the Goldman Sachs report (which looks at O*NET’s inventory of the knowledge, skills, and abilities required for hundreds of different occupations)14 with the constantly updating nature of a prediction market? What if any unexpected delays or sudden leaps in progress could be better contextualized? Imagine, if you will, a site like willrobotstakemyjob.com, where you could see links to AI’s latest progress on various tests relevant to each occupation, make bets (a la ’s Robots-Took-Most-Jobs insurance?), and discuss? I find the work being done by Stanford’s HELM team, the BEHAVIOR benchmark for robotics, and this research that measured how much use of ChatGPT augmented workers across a variety of tasks very intriguing, and would love to help realize a multidisciplinary project like this! 15

Understanding ourselves better to keep AI safe

As we continue to make progress in assessing AI’s capabilities, I expect we’ll continue to reveal unexpected insights about ourselves, and what makes us “intelligent”. Many new capabilities that are developed may be double-edged swords in that there is also potential for significant harm, whether from technological unemployment, algorithmic bias and other ethical considerations, or the potential to aid bad actors. The debate over which of these threats among others are the most salient is hotly contested, but it’s clear that regulation of AI will need to span many domains. Better tools for tracking AI progress could help us understand the comparative advantages we are likely to continue to maintain as humans, and make this regulation more attuned with the reality of possible threats.

4. Algorithmic efficiency

Where accuracy is defined, “algorithmic efficiency” can be calculated. Processing power and model size are fairly straightforward measurements, but if we were to imagine how much accuracy might still improve even if these didn’t change, we could calculate how much more efficient AI has become. Epoch AI also did an analysis of this and the results are impressive— every 9 months, the introduction of better algorithms contribute the equivalent of a doubling of compute budgets, at least for image recognition.

But “algorithmic efficiency” is a nebulous term. Below are five categories I’ve found helpful for classifying the various gains in performance we might see from aspects of AI not related to better hardware, training larger models, or using architecture that is fundamentally different from current transformer models (fundamental changes in architecture, like the shift to transformers that occurred in 2017, could be considered a sixth type of “algorithmic improvement” to look out for16):

Pruning data and sparsifying parameters

• Pruning data: The quality of data that a model is trained on may vary considerably. An appropriately curated subset of that data (as long as it has sufficient diversity in terms of subject matter) can lead to similarly accurate answers but require much less computing power, such as this one that achieved similar accuracy after pruning 25% of its original dataset, and revealed some insights into which examples are best to keep based on a dataset’s original size. Given the billions of data points that are generally involved in training a model, AI naturally assists with this pruning.

• Pruning parameters: Alternatively, creating a “sparser” model that can selectively ignore parameters that are less relevant to each batch of data, or route them to an “expert” that specializes in a subset of them (as GPT-4 does) improves efficiency in the other compute-heavy dimension of a model. For example, in March 2023, SparseGPT revealed little loss in accuracy even after roughly half of its parameters were effectively pruned, and WANDA, an even more economical pruning approach was proposed three months later.

• A mixture of both: Focusing a model on a specific domain with high-quality inputs can reduce the compute needed for both dimensions (data and parameters), as this model 8,000 times smaller than ChatGPT’s free tier performed just as well or better in the specific domain of answering Python coding questions. There is a strong argument here though that data-driven (rather than parameter-driven) improvements will have the most impact.

Fine-tuning

• Layers: If we can imagine the number of data points and parameters as the length and width of a model, we can think of the number of layers it uses to transform an input into a desired output as its depth. A model that gives reliable answers in one domain can be fine-tuned for another one by peeling back and retraining some of its layers closest to the output with data related to its new task.

• Attention: Different “attention heads”, which are each tasked with looking out for certain subtleties in the use of language, interpret the input and transform the potential output at each layer, and can be adjusted as well. There is a limit to how much content these can ingest at a time, though developments like the “Hyena operator” may allow for a model to be able to take in a larger context window, which has implications from improving personalized AI to long-term planning to solving more advanced problems in genomics.

• Different methods for fine-tuning: A model can be fine-tuned on a new dataset that is more specific to a new task it is expected to accomplish. In some cases, this can be done by simply giving the model some specific instructions. Otherwise, using interactively-generated data coming from human annotators, and providing the model with feedback on its responses can be a powerful but labor-intensive approach. This last method is known as Reinforcement Learning with Human Feedback, or RLHF.

• “Self-improving” AI techniques: Fine-tuning can be automated such that AI is able to improve itself, though this does involve some structure beforehand to prompt the model to ask itself questions that include example answers (AKA a “few-shot”17 approach) and instruct the model to provide its answers in a way that spells out each logical step (AKA chain-of-thought reasoning). These are also techniques that help people get higher-quality answers from AI, and can be used across different domains— similar self-improvement has been seen in robotics.

• How much self-improvement are we actually seeing? The optimistic case for self-improvement is that models appear to be at or above human level for many simple data curation tasks. The pessimistic case is that models may not be as successful at curating the data needed to accurately perform more complex tasks, and that human error and bias in annotation might constrain progress even in manual improvements. A list of other risks of self-improving models can be found here.

Interpretability features

• For all of the impressive features that 2023 AI models have, interpretability, or the ability to understand why a model made the choices it did, is one of the biggest shortcomings. ChatGPT doesn’t cite its sources, and even tools like perplexity.ai only provide links without going into more detail as to why those sources were used and not others.

• The number of layers and attention heads make a model’s decision-making exceedingly difficult to disentangle, which can make it harder to trust the outputs. This makes interpretability one of the more labor-intensive and subjectively evaluated areas of AI research in 2023. You can get a real sense of the complexity of this field from reading this get-started guide.

• Breakthroughs in interpretability might be heavily incentivized particularly in fields like medicine where AIs might eventually perform better than human doctors, but still make errors that will need to be explained. There is also a high level of interest in interpretability from junior researchers in the field, and over 1,000 papers are published about interpretability each year.

• Interpretability may unlock other benefits as well, though some of these are contentious, and there may be a long way to go to get there.

Multimodality

• One of the aspects of current advances in AI that has surprised experts the most has been how transferable LLMs’ abilities are across domains. This aspect (called “transfer learning”) can greatly accelerate progress by making it easier to fine-tune existing models, and allowing for machines to understand plain-English instructions rather than requiring other specialized code. Here are a few of the most significant examples:

  • Translation between languages: LLMs can understand non-English languages relatively well in spite of these languages making up only a tiny fraction of their training data (which is still mostly in English). As of 2023, they are better at translating into English than the other way around, though they have not yet achieved the level of accuracy of professional human translators.

  • Understanding similar concepts across text, code, and images: In May 2023, Meta released ImageBind, foundational code which would allows for translation across different modes of communication entirely: for example, a short video of a train could be generated based on the sound of a train horn. An impressive example of a 100% AI generated short video from July 2023 can be see here— check out the description to learn more about the tools used for the story, images, video, narration, sound effects, and subtitles.

  • Moving around in the real world: an AI system that can translate between modes and follow natural-language instructions can go from navigating itself around in a virtual environment to moving around in the real world. This multimodality brings further advances in self-driving cars as well as other kinds of robots. In July 2023, a new way to translate images and language into robot actions was developed that allowed a robot to achieve a similar level of accuracy as one that was trained on over 10x as many parameters just a few months earlier, though like other systems it still struggled with commonsense reasoning.

  • Medical applications: there are even more streams of information that AI models can process in the medical field, from interfacing directly with the brain to proposed moonshot projects like modeling every single possible molecule for drug discovery and building an AI model of the human body down to the cellular level. Important new breakthroughs are regularly published here, and range from specific applications18 of AI to new tools to expansions of these capabilities from one tool to another to new systems that are generally knowledgeable across domains. While some of these systems might be just specialized versions of ChatGPT, Foundation models for Electronic Medical Records (FEMRs) are another important variant. FEMRs use longitudinal health data to make predictions about a patient’s prognosis. And beyond medicine there are many other applications that scientists in general are both excited and concerned about.

Human skill at using AI

While human skill isn’t an algorithmic improvement in the strict sense, as people get better at using AI-powered tools, it will appear as if their performance is improving as well.

• The starting point: Just as people a generation ago didn’t become proficient at using computers overnight, there is a learning curve to making the most out of current AI technology too. As of July 2023, only 24% of Americans who’ve heard of ChatGPT have used it, and an even smaller percentage have used it for work or learning. There is still a long way to go here, though adoption has been a lot quicker than with previous technologies.

• Prompting skills: Techniques used by AI researchers to get better results when testing their models, like providing examples of answers to similar questions, or simply asking the AI to “take a deep breath and think step-by-step” can help everyday users get better results as well. shares some even more sophisticated prompts here— imposing some constraints and providing more detailed information about the intended audience can greatly improve the quality of the response19.

• Imagining where AI can be applied: As we explored in the above section about multimodality, AI tools aren’t just chatbots! From image generators to robotic lawnmowers, the many use cases for AI will challenge our imaginations for years to come. has done some interesting conceptual work in this area, envisioning how we might make use of AI to challenge our assumptions and get better feedback on our ideas20.

• Baking AI into everyday products: from the imaginations of designers, we can expect to be deluged in the coming years by AI-powered products both useful and overhyped. “Copilot” tools have already been used in some areas to great effect. Depending on the metric used, GitHub Copilot has increased the productivity of software developers, particularly more junior ones, between 25 and 55%21, and 75% of its users felt more satisfied. Similar tools are rapidly expanding to other sectors like sales, marketing, and customer service work with Salesforce’s Einstein Copilot. An expanding number of AI-connected extensions22 bring these tools into browsers, and even operating systems are getting an upgrade. Meanwhile, ChatGPT’s functions are expanding thanks to an emerging universe of useful plugins, including Advanced Data Analysis (formerly know as Code Interpreter). There are some excellent examples of how to use this here— I expect this tool will help me finish my upcoming analysis of the BLS’ latest employment projections much more quickly than I’d originally expected23.

• Meta-skills: As AI tools get better, it’s important to be aware of what tasks AI isn’t an appropriate solution for, and recognize where reliance on automation might atrophy our own skills. These are important concepts to teach AI agents as well. A recent collaboration between Google and Deep Mind explored when a tool might accept an AI-generated diagnosis vs when it might ask a human professional. But quantifying the uncertainty and knowing what an AI might get wrong isn’t easy. There is a quiz you can take to guess if GPT-4 got various questions right, and it is surprisingly difficult to do well. My hope is that with a better way of assessing AI’s abilities relative to ours in real-time, we will also become more knowledgeable in this area.

It matters where AI improvements come from

If the most dramatic improvements continue to come from easily sharable techniques for more effective use of data and parameters, and models continue to surprise researchers in their abilities to adapt to novel modes, powerful AI is less likely to be constrained by computing power and the cost of human labor. This has sparked two major concerns:

1. Concerns from investors that large AI companies may have less of an advantage than they had originally assumed.

2. Concerns from security experts that small-but-powerful open source AI tools could be harder to regulate and make it easier for bad actors to wreak havoc.

As an example, the computation power alone to train the premium GPT-4 is reported to have cost between $32 million and $63 million. But it’s important to note this is likely only a fraction of OpenAI’s operating expenses, which totaled over $540 million in 2022. While its energy costs are likely still under $10 million/year according to these figures, many of its employees (375 as of early 2023) make nearly $1 million/year, and much of the remaining spend likely goes toward outsourced contracts for data annotation24. As much as compute might be bottlenecked, it seems like skilled AI researchers are in even shorter supply25 as labor still appears to be the biggest part of the cost of training a state-of-the-art LLM.

If this continues to be true, we might expect the trajectory of AI to be rather similar to ordinary software, with open source26 tools like Facebook’s LLaMA and its descendants being used for some applications and closed-source tools being used for others. Still, a key variable to follow over time (and something that should be more robustly tracked) might be the gap in performance between open and closed source tools as this summary from outlines. Such a variable, being inclusive of AI’s overall performance and usefulness would heavily influence the nature of our fifth big variable to watch: investment.

5. Investment

At a high level, global corporate investment in AI increased 5-fold between 2017 and 2021. There are a few parts to this story, including a preface about the scale of these dollar amounts:

• The start of the boom and the 2022 slowdown

• 2023’s growing clash between giants

• What can the market tell us about the future of hardware, software, robotics, and the economy as a whole?

The scale of investment in AI

Staggeringly large numbers, whether exaFLOPs or billions of dollars can be hard to imagine without better context. US private investment in AI in 2022 was $47 billion. That’s equivalent to:

• About 8% of all US software investment

• 1% of all US private investment

• 0.2% of total US GDP

0.2% of GDP is still a relatively small sum compared to how much we hear about AI, and how much it might impact daily life. But by 2032, this Bloomberg report suggests that the revenues of the generative AI industry could be as high as $650 billion in the US alone27, or $1.3 trillion globally, which would be close to 2% of 2032 GDP (assuming a continuation of average economic growth, though US investors appear to expect more). Forecasters of total global corporate investment in AI on Metaculus make a prediction similar in scale.

How does this compare to historical trends?

US private investment in software passed 2.3% of GDP in 2022 and has continued to grow from there. To grow by an additional 2% of GDP within a decade would be unprecedented— even the software boom in the 1990s only amounted to 0.75%, or 1.3% if you include hardware equipment as well.

Knowing how useful AI copilots have already been for software developers though, we need not think of AI as entirely separate— it will almost certainly “eat” a certain amount of software spend such that the overall increase might only be somewhat higher than what we saw in the 1990s. And there are many interesting comparisons one may draw here. As puts it:

The internet blew open the gates of distribution

Generative AI blows open the gates of production

It took time to build the infrastructure of the modern internet, teach people how to use it, and eventually change habits (for better or for worse) such that the average American spends 6.5 hours per day looking at internet-connected screens. This infrastructure both physical and cognitive is what has allowed AI tools to be adopted at record-breaking speed. That the network for distribution is already in place is what allows for more aggressive predictions.

The start of the boom and the 2022 slowdown

Earlier, we saw that the largest AI models grew 100-fold between 2019 and 2023. This was enabled by the five-fold increase in investment that happened 2 years earlier between 2017 and 2021, as we saw in the first chart of this chapter. The extra investment was complemented by a tripling in price performance in that same time, just a little slower than Moore’s Law.

3 times 5 doesn’t add up to 100 of course, so it’s notable that the concentration of investment in computing power also increased. We see this in GPU-maker NVIDIA’s revenue, which also tripled from $9 billion to $27 billion between 2017 and 2021, largely from sales of its chips to data centers specialized in training AI models.

Similar to the dip in overall investment, NVIDIA’s revenue also dropped slightly in 2022 as higher interest rates took hold and demand from cryptocurrency mining took a hit, but Q2 of 2023 saw a record increase as investment accelerated again.

2023’s growing clash between giants

Just as rate increases slowed in the spring of 2023, the apparent value of generative AI was enough to motivate an even larger wave of investment in hardware than before even as venture funding continued to decline. While Sam Altman of Open AI stated that it would be “some time” before we see GPT-5 in June 2023, just one month later, the company filed a trademark for it with the US Patent Office. Google’s Gemini, which could be at least 5x larger and more multimodal than GPT-4 is expected to be released in December. This could put some pressure on OpenAI to roll out something new if they haven’t done so already by that point.

This competition between OpenAI/Microsoft and Google will be interesting to watch. While visits to ChatGPT have plateaued since May 2023 (and fallen a bit during the summer while schools are out), it still gets about as many visits as Netflix, more visits than any single news website in the world, and over 10 times as many as Google’s current competitor Bard.

But Google Search still gets nearly 50 times as much traffic as ChatGPT. If their sizeable cash and data advantage allows them to provide useful features to their large user base across their many products (and find a way to make LLM-powered searches economical) it will change the trajectory of AI.

Of course, these companies are only two in the wider universe of tech. It’s important to note that not all areas of AI investment declined in 2022. Indeed, there are many use cases such as cybersecurity, industrial automation, semiconductor research, and drones that are just starting to take off.

What can the market tell us about the future?

NVIDIA’s stock price grabbed headlines in mid-2023, and OpenAI’s $80-90 billion dollar valuation also turned some heads. The valuations of both of these companies— one hardware and one software— tells us a lot about both the excitement and concerns of investors. Thinking about growth in these industries can also help us better contextualize the progress in robotics.

Hardware investment

On the hardware side, as of October 2023, NVIDIA’s stock price stands at over 100 times its total earnings per share over the previous year (AKA a P/E ratio of over 100), or about 4x higher than the market as a whole. Of course, its frothy valuation is based on its Q2 earnings, which represent a significant jump in its data center sales.

Many analysts expect these to grow as time goes on. But even if we just held Q2 earnings as they were for the rest of the year, a $432/share price puts its forward-looking P/E ratio at a much more reasonable 4028.

Looking at things this way, it seems like NVIDIA’s earnings would only have to double in order to get back to the forward-looking market average. Given various growth projections from 23% to 74% annually, this could be achieved in 1-3 years. But there are a few risks to look out for:

• Demand for chips: this is contingent on AI models continuing to scale up as we explored above. The optimistic case is that as models move from text to the data-heavier formats of image and video, more compute will be needed. Image generator Midjourney’s ambitions include generating real-time video and even custom video games, but their CEO has indicated that more computing power is needed to achieve this. Tesla plans to build a computing cluster more than 10 times larger than OpenAI’s by 2025 to more extensively use the millions of hours of driving videos its cars have captured over time for training purposes. It’s important to recognize that inference costs computing power as well as training, so any significant user growth would imply a need for more computing power as well. Recognizing this source of demand, NVIDIA’s upcoming GH200 GPUs will have 230 times the GPU memory, and will be even more specialized for inference than the current H100s.

  On the flipside, Midjourney has been able to create a very impressive tool for a model that only costs $50,000 per training run, and if Tesla is able to bring full self-driving cars to market based on its 2025 investment, it’s not clear how much more computing power it would need after that. Economy-wide, it could be that all it takes is a spike of initial investment (in GPUs in this case) to make meaningful increases in productivity, as was the case for both electrification in the 1920s and PCs/internet in the 1990s.

• Broad vs narrow demand: will growth in demand for computing power be dominated by a few leaders, or more broad-based? Looking at the Top 500 supercomputers, growth in top computers appears to be outpacing the growth in the 500th fastest one, but this could be an informative difference to watch over time, albeit a lagging indicator.

• Competitive pressure: the rush to buy GPUs has brought NVIDIA’s profit (net) margin from 19.2% to 50%, but this is unlikely to hold over the long run. Retaining the overall level of earnings at the likelier long-term margins means revenue would have to at least double, if not 10x, unless NVIDIA’s “moat” is wide enough to keep both its high margins and dominant market share (91.4% of the enterprise market in 2021). The margins of the semiconductor industry on average are only 5.6% which could be a risk if other companies become competitive (including both NVIDIA’s OEMs and even their customers), or if there are breakthroughs in new computing paradigms that NVIDIA is not invested in. On the other hand, NVIDIA’s R&D spend is nearly 20% higher than its closest competitor AMD, even after accounting for gross revenue. This budget may allow them to continue to maintain their advantage in specialized improvements that sparsify parameters at the hardware level, attacking the memory bottleneck mentioned earlier.

Altogether, it seems like a bet on NVIDIA in the $430/share price range given Q2 2023 earnings is a bet that recurring demand for computing power will grow between 5x and 20x and remain there, on top of the 10-20x predicted improvements in price-performance over the following decade. This is roughly in line with Metaculus forecasts for the largest as well as the most expensive training run, both of which include considerable uncertainty29.

Software investment

On the software side, OpenAI has been reportedly valued at $80-90 billion in spite of only a little more than $1 billion in revenue and $540 million in expenses, implying an even higher P/E ratio near 200. How can it be worth that much?

It’s important to remember that growing software companies are often valued much higher than companies that manufacture physical components. Additional copies of software can often be sold at very little additional cost after an upfront investment, and customers might not be able to switch so easily once a solution is chosen. As an example, Salesforce’s P/E ratio has frequently been north of 100, and now they have a major AI product as well. Broadly though, considerations for investors in AI software are similar to those for AI hardware— demand and competition:

• Demand for large AI models and related services: For this, we may also want to think back to the Bloomberg prediction. If generative AI revenues do indeed grow 10-fold relative to the overall economy by 2032, a 200 P/E ratio shrinks to a 20 (below the market average of 25) in just 9 years, though only if the profit margins remain the same. Given the many applications of AI in software development itself, it’s possible that software may become significantly cheaper, increasing demand but also competition. On the other hand, if it becomes significantly harder to achieve a high enough level of accuracy for more advanced applications, the demand for generative AI in general could begin to stall. An early indicator of this might be the slowdown of new ChatGPT Plus subscriptions in September 2023.

• Competitive pressure: as we explored above, if open-source models can be competitive with state-of-the-art ones, knowledge about how to best deploy AI for each use case could trump having a big general model. Those who believe this tend to invest in smaller companies rather than making investments in large ones. Midjourney is an example of a company that has already generated $200 million in revenue as of September 2023 with 40 or possibly even fewer employees with no outside investors. But as the competition between Google and OpenAI shows, having a large user base and a first-mover advantage can also be decisive.

Robotics investment

It seems like we hear a lot about AI, but not as much about robotics in 2023. Compared to 2022’s global private investment in AI of $85 billion and revenues of $40 billion in 2022, what does the global market for robotics look like?

The International Federation of Robotics (IFR) collects numbers on robot installations, but there doesn’t seem to be as centralized of a source for how much is actually invested in the industry, at least globally30. Based on information from a few different reports which I’ll reference below, my best estimate for this is $69 billion in 2022, just a little less than AI. Growth projections are also slower at 17% annually which would put the total market at $286 billion by the early 2032, only 1/4 of the forecasted size of generative AI market. That said, within robotics there could be some breakout sectors. Let’s take a closer look at the three major categories in this industry:

• Industrial robots: with 553,000 units sold worldwide in 2022 at an estimated total cost of $26.5 billion, each robot cost $50,000 on average31. This was a 5% increase over the year before, much less than the 10% that was predicted previously. It’s important to note that more than half of all industrial robot installations happened in China32, and there was a big jump there in 2021. However, with manufacturing construction picking up again in the US, might more robot sales soon follow? The IFR’s World Robotics 2023 report estimates annual global growth going forward at 7%, but another analysis that was published by Bloomberg suggests it could be as high as 10.5%.

• Professional service robots: this market includes robots in the fields of transportation & logistics, hospitality, healthcare, professional cleaning, and agriculture. There are some excellent visual examples of these in IFR’s 2022 report. While the absolute number of new service robots installed worldwide in 2022 was just over a quarter the total of industrial ones at 158,000, the total cost of these came to $21.7 billion33, and grew by 48%, after 37% growth in 2021. Seeing such growth in spite of the $137,000 average unit price makes me wonder if the 21.6% growth forecast for this sector is too conservative.

• Consumer service robots: these include the sorts of robots you might buy for work and fun around the house, from Roombas to recreational drones. It is the smallest market financially, at $10.4 billion in 2022. However, it is the largest market in terms of number of total units at 19 million for vacuuming and lawn-mowing robots 2021. While the growth projections from Statista seem to suggest this market is cooling off to growth rates around 5%, it will be interesting to see if recent advances in multimodal AI make these sorts of robots useful and/or cheap enough for more people to want to buy them.

While the market for GPUs is dominated by NVIDIA, and the market for generative AI is associated with several well-known companies while everyone else figures out how to offer AI themselves, the robotics market in 2023 is much more fragmented. Overall, it seems like there are 5 different types of companies in this space:

• Companies with long histories in the automation space like Omron, an early manufacturer of electronic ticket gates and ATMs

• Companies who make industrial robots as well as many other types of electronics and related components like ABB

• Other established companies like Intuitive Surgical that make robots for specialized use cases in healthcare

• Robots-as-a-service (RaaS) companies that allow operators to test robots out before committing to a larger purchase

• Startups working on a variety of specific challenges (2013-2016 were boom years in the robotics startup world)

If growth in service robots continues its recent trend of 40%+ each year, it would bring an additional 15 million new robots online by 2032, and the total robotics sector to over $700 billion, over half of AI’s projected revenue instead of just a quarter. Such growth would beat market expectations— between 2019 and 2023, robotics-focused funds tended to slightly under-perform the S&P500 unless they had heavier exposure to AI-focused stocks. Could this be an example of investment in AI “crowding out” robotics? And is the 66% growth in the RaaS market a leading indicator of more companies trying out robots for their operations?

As we explored in the earlier section about multimodality, AI and robotics are likely to see even closer synergies in the years to come. However, there are still quite a few unsolved challenges that this detailed review of the literature on robotics explores. This is something I’m looking forward to diving into deeper in a future piece.

Investment: the bottom line

In an economy where interest rates are higher than they’ve been since the 2001, investment in technology is quite different than it was in the 2010s. While Amazon was still consistently losing money in most quarters as late as 2013, both OpenAI and Midjourney are well-known AI companies that are already profitable in 2023. With higher interest rates, venture funding is down, and companies have to prove their value more quickly than they did a decade before. Yet interest rates were similarly high in the 1990s and the dotcom bubble still happened. And it’s important to mention that not all AI ventures are profitable— Microsoft’s GitHub Copilot reportedly loses more than $10 per month per user. How can we assess if the stock market overall is fairly valued or in a bubble based on the promise of new technology?

While the early beneficiaries of AI have been leading 2023’s stock market gains as one might expect, a chart that really caught my eye was the one below that compares the overall S&P 500 forward P/E ratio (a possbily risky asset) with 10-year inflation protected yields (TIPS, a very safe asset).

One of the axes is flipped, so we see that the trends are a close inverse correlation with one another. There was an equilbrium in the late 2010s where forward P/Es were around 17 and TIPS were around 0.5%. TIPS fell to -1%, with stock valuations conversely rising about 30% to 22 during a pandemic-era equilibrium. After a brief period in mid-2022 where the pre-pandemic equilibirum was restored, these trends started sharply diverging from each other in late 2022, around when ChatGPT was released. Now, TIPS have risen 1.5%, but rather than fall by 30% toward a forward P/E ratio of 12 as the trend might have suggested before, stocks have continued to rise toward 19!

If we take the P/E ratio of the previous equilibria to be 5.8% and 4.5% respectively, and compare that to the TIPS of 0.5% and -1% respectively, we see that historically, investors expect 5-5.5% additional returns for the risk of investing in the stock market. With TIPS approaching 2.5% in October 2023, that means investors are expecting 8% returns from stocks over the next 10 years, or 2% higher earnings than usual. This is on the high end of the 0.3% to 3% range of additional annual productivity growth that the April 2023 Goldman Sachs report suggested. As economist points out, even this additional 2% of growth per year that the market has priced in is historically exceptional.

If you were to invest just in the stock market in general given this information, what would you do?

If you believed that AI would lead to no greater than normal growth, you might think that we are in a bubble and the stock market could be as high as 50% overvalued given the relative safety of inflation-protected Treasury bonds. On the other hand, if you believed that AI might someday bring us anywhere close to an unprecedented rate of 30% annual GDP growth34, you might think that the current market is still quite a bargain. As with many of the predictions we have explored so far, there is quite a range of uncertainty. Reading through detailed reports like the Goldman Sachs and McKinsey ones, and comparing their predictions with your experience I think is the best way to assess if the market (or any individual investments you might be considering) are fairly valued35.

While the financial markets can give us a good indicator of what wealthy investors think will be valuable, and observations about computing power and algorithmic improvements can give us a sense of how the capabilities of new technology might evolve, it’s important to not treat these forecasts as wholly deterministic. Ultimately new technologies will ideally be shaped to serve the common interest based on public opinion and regulation, our sixth and final variable.

6. Public opinion and regulation

As we wrap up this deep dive into AI, it’s helpful to look back at the other key variables we’ve considered and how public opinion and regulation impact each one:

• Increases in computing power have been fairly predictable for the past 60 years, but the threat of geopolitical conflict looms over the industry. The strategic value of semiconductors has led to export restrictions and new industrial policies designed to secure a reliable supply. Some have also argued for computing capacity restrictions for individual actors.

• Increases in the amount of data that’s collected (and later fed into AI models) can be modeled, and the contents of what is stored will depend on privacy and copyright laws. Beyond legal restrictions and questions about what is considered “fair use” of copyrighted work, individuals and companies are also moving to limit the publication of their own data into the public sphere. Larger AI models will naturally attract more scrutiny.

• We should expect the software behind AI to continue to become more powerful, though as we explored above, there are many worries that open source AI tools will be difficult to regulate.

• AI’s abilities continue to improve in many areas, though there are some notable weaknesses. Regulations about the sorts of decisions that AI systems are allowed to make play a role here, as do keeping a close eye on any externalities that seemingly competent systems may introduce, from traffic jams to increased cybersecurity threats36. Intelligent regulation will rely significantly on lawmakers’ understanding the current and likely future state of AI abilities, and there is naturally a great deal of concern here about the future of employment.

• As the level of investment in AI increases, so to will opportunities increase to regulate operators above a certain size. Governments might also change the amount they invest in R&D in the field as they investigate their own AI use cases.

However, there is one type of regulation that doesn’t fit into the above boxes: rules and norms that govern accountability, ownership, and agency.

Accountability, ownership, and agency

Who is accountable when problems arise? As AI-powered systems move from generating funny images of cats to making life-and-death decisions on the streets, this question grows more important every day. The number of AI-related legal cases continues to grow rapidly, and cases about the liability of self-driving cars might inform how liability is settled in other cases of embodied automated systems.

Who owns the intellectual property that autonomous systems create is another matter of intense debate. In 2023, the US Patent Office has ruled multiple times that AI-generated artwork is not eligible for copyright, but makes a strong case here for AI-generated artwork being analogous to photography (which after some legal battles when it was a new technology, ultimately became eligible for copyright in most instances). Significant changes in the Patent Office’s ruling would have a big impact on the business applications of generative AI.

The accountability of AI systems, as well as people’s level of trust in them, will influence the overall level of agency these systems are accorded over time. AutoGPT, an AI application that you could use to, among other things, order a pizza was released to much fanfare in April 2023. Though interest has declined since then, frameworks like LangChain have been developed to allow applications that use AI to cross various modalities and better “remember” what they need to do across chains of tasks. As AI systems are increasingly connected to eyes and ears, it would be interesting to measure how much money they have direct access to over time as a barometer of the public’s trust in this technology. If such a system does something unintended, when is it the fault of the user vs the fault of the developer? Vicarious liability theory, and considerations about who is responsible for each layer of an AI are relevant for these sorts of questions. There is an excellent summary of these concepts here.

In imagining how complex agentic systems deployed on a broader scale might play out, the history of algorithmic decision-making assisted by machine learning in stock trading could be an instructive example. These systems already account for a vast majority of daily stock trades, have an extensive history of regulation, and have already had to deal with challenges like being vulnerable to misinformation on social media. An important question to account for though, is how well do systems whose use is regulated within a company scale down to personal use cases?

To help us visualize what a more automated future might look like, I’ve proposed a heuristic similar to the classic “Eisenhower Matrix”.

A way to think about automated decision-making

Routine, less consequential decisions are easy, and many of them (like spam filters or autocomplete) have already been seamlessly integrated into daily life. Recent advances in AI have allowed for the possibility of non-routine decisions to be automated as well, but the “squishiness” of models and the difficulties they often have in providing up-to-date factual information mean that there is some risk in automating any non-routine task.

The line between routine and non-routine tasks can be blurry in highly consequential use cases like self-driving cars. If a self-driving car has training on how to handle 99.999% of situations, how does it handle the 0.001% of situations that are truly exceptional? It’s important to note that both of these axes are spectrums, and what seems non-routine today may become more routine later when more data is collected and the developer of the AI system is able to sufficiently assume liability for any harm that might be caused. Liability for the more consequential use cases naturally needs regulation to function, and the EU AI Act’s risk-based approach (in final negotiation as of October 2023) includes tiers of risk (limited, low, high, and unacceptable) that roughly correspond with the four categories of this automation matrix. However, it’s important to note that some tasks that could be routine but would be highly consequential if widely done by machines (like facial recognition and its dangers) might fall into the “Unacceptable risk” category.

Public opinion and trust, perceived benefits and risks

Regulation and public opinion ideally go hand-in-hand. How are public opinion, public trust, and social norms around emerging technology measured? How does the public perceive both benefits and risks from AI and robotics, and how much does that reflect or differ from the range of opinions expressed by experts?

Pew Research has a whole category of their research dedicated to tracking public opinion on AI. A poll they conducted in the summer of 2023 found that the American public tends to be more concerned that excited. Opinions differ elsewhere in the world though— public opinion across eastern Asia has historically been much more optimistic. Considering that there is more investment in AI in the US than anywhere else, does this imply we might see a shift in AI investment from west to east in the coming years? Curiously, a similar difference in attitudes between these two regions (exacerbated by a terrible accident in the year 2055) appears to be the premise of the movie The Creator.

Of course, as this movie shows, public opinion about new technology can shift rapidly in response to current events. It is interesting to consider what benefits and risks people in different parts of the world are most excited and concerned about, and this Ipsos survey in the summer of 2023 asked just that. It found a big shift compared to 2021 toward agreement that products and services using artificial intelligence will profoundly change daily life in the next 3-5 years. It found an even bigger shift toward people agreeing that such products and services made them nervous.

In terms of aspects of AI that people were most optimistic about, saving time, improving entertainment options, and improving health featured high on the list.

Other improvements related to jobs and the economy were more controversial. A similar pattern to other global surveys held in that the US and other Western countries were more skeptical. India, southeast Asian countries, and Latin American countries were more optimistic, though the survey tended to oversample more online, wealthier individuals in these areas.

In considering the risks of AI, this survey mostly focused on the economy and the job market, but it did also ask questions about trust in AI companies to protect user privacy and not show bias. The patterns of optimism and skepticism in responses to these questions are similar.

The difficulty of weighing AI risks against each other

One thing the Ipsos survey did not ask is to have people rank their top concerns about AI. From the growing database of AI-related incidents to more theoretical concerns about AI posing an existential threat to humanity, considerations about what risks warrant the greatest share of our limited attention are hotly contested among the various factions of experts. But if measuring AI’s abilities and predicting when developments will occur is already difficult, assessing the scale of threats compared to when they might occur (short-term vs long-term) is even harder for an average person to assess.

In future pieces, I’d like to do some deeper dives into AI and robotics capabilities, including some more specific predictions that could help better contextualize a later exploration of the various risks. I haven’t yet seen a good data-driven piece that contextualizes the various threats in the way that I am envisioning, but will update this section if I do encounter something. In the meantime, has published a few pieces that have helped me think about high-level taxonomies of AI risks, and the improbability of complete catastrophe. AIRS has proposed another more detailed taxonomy of risks, and Stanford’s annual HAI report also includes robust metrics of ethical considerations, representation in the field, and the risks that companies consider most important.

Keeping up with AI regulations

Among many other things, Stanford’s annual report also includes a detailed section about the regulation of AI. The one drawback is that news in AI moves fast, and while regulations move somewhat slower, there are more up-to-date, comprehensive sources of information elsewhere. While major changes in regulation are likely to be covered by major news outlets, what about changes in policy that fly under the radar?

Privacy-focused groups have us covered here. The IAPP provides periodic summaries of the state of regulation at the national and international level, while EPIC has coverage of U.S. state level regulations. If you’re specifically interested in regulations about how the liability of self-driving cars are assessed, this Wikipedia page is regularly updated across many jurisdictions. The only caveat is that these summaries aren’t easily filterable if you want to look at trends in a particular type of regulation across jurisdictions, though perhaps this is something that applications of AI might make easier? It’s interesting to imagine an AI assisting with the categorization of these regulations across various taxonomies in closer to real-time as regulations are negotiated and passed, though a matter as important as coverage of regulation would also likely continue to require human review before publication!

Wrapping it all up

So there you have it! You should now have a good mental model for these top 6 trends in AI and robotics. If you’ve made it through all of it, thank you! I hope my explanations help you connect whatever you might read day-to-day to the bigger picture. This analysis ended up being a lot longer than I’d originally anticipated when I first started writing it in July of 2023, and there were definitely some more examples and details I wanted to dive into, but cut for the sake of brevity.

If there was one variable I had to pick to study more, it would be accuracy and alignment. The future trajectory of this is the biggest unknown even among experts. Epoch AI, whose research I’ve cited extensively here, has come up with a sandbox for people to model when transformative, human-level AI might arrive. These variables in this sandbox roughly line up with the ones outlined in this piece, with the following clarifications:

• The variables related to computing power and algorithmic progress line up just about exactly.

• The investment related variables assume investment translates directly into model size.

• There is no equivalent variable correlated with public opinion and regulation37.

• Their “compute requirements” variables are the most abstract, but they correspond with accuracy and alignment and are explained more in their paper on the Direct Approach.

Epoch’s Direct Approach effectively hypothesizes that for every 6 orders of magnitude (OOM) that effective FLOP (model size + algorithmic improvement) is increased, the ability of AI also increases by an order of magnitude. But what does that mean in practice? It does NOT mean that an ability where AI is at 80% human level would be at 800% of human level, but rather that the context window is extended. In other words, a model that performs 10 times as well can generate an output that is 10x larger at a similar level of accuracy. So, if today’s state-of-the-art models can write human-level blog posts, this theory suggests that a 6 OOM increase in FLOP would put us at human-level scientific papers, and that length (the K-level) might be enough for AI to radically transform the economy and society. If we assume that hardware will grow by 1 OOM in 10 years, as will investment, and algorithmic improvements grow by an OOM every 2.5 years, that puts us at 6 OOMs of additional effective FLOP within a decade.

Of course, to say that state-of-the-art models can write a human-level blog post, or its equivalent in images or video isn’t necessarily true in all cases. This is where the “Human slowdown factor” comes in. You can think of a lower value here as AI’s performance at a task where a high level of accuracy is required. Maybe ChatGPT can produce average human-level writing for the length of a blog post for an average task, but would struggle to achieve a more exacting level of accuracy for anything longer than a tweet in other domains.

As helpful as this framework is for thinking about these concepts at a high level, the assumption that accuracy will continue to improve at the same rates per unit of computation as has previously been observed might not hold to be true, or may only be true for some abilities but not others. Alternatively, there could be further surprises ahead— perhaps some new technique, or a critical level of quality data unlocks abilities not previously observed. This is yet another argument for a more robust and transparent method of tracking AI capabilities.

The early 2020s have been full of surprises in terms of AI. But amidst the excitement, there are also some signals to be cautious about. It might be difficult to continue scaling models as quickly as was done in 2017-2023. If large context windows are important for more AI functions, what happens if their performance continues to be less than advertised? If there is a hype cycle with every new technology, is AI really that different?

If the real world continues to be messier than models suggest, I wouldn’t be surprised if we find ourselves in a “trough of disillusionment” at some point in the middle of the 2020s. In spite of this, I think the multimodality of AI and robotics technology has tremendous potential. I am optimistic we will see medicine improved and drudgery reduced. At the same time, I think the likely future path of automation means that jobs remaining for humans may be ones that require more intensity of judgement. Alongside this, there is a concerning trend towards more surveillance, both in the workplace and elsewhere. We will need to be vigilant to ensure gains from new technology are distributed in a way that benefits collective well-being. The impact of social media on mental health and politics is a cautionary tale. A question we should ask ourselves more, as individuals, as people interested in technology, and as participants in the public sphere: how can we spend more time doing what we love? How can we make this dream possible for the greatest number of people?

Acknowledgements

Writing this piece wouldn’t have been possible without the many experts out there who have generously shared their insights freely and publicly. I’ve learned a lot from them and I hope you can too. I’ve cited these folks extensively, but I would like to give another shoutout to:

The Epoch AI team (, , among others) for their detailed analysis of high-level trends.

Dylan Patel’s for his team’s deep insights into the trends affecting hardware manufacturers.

Timothy B Lee’s for his deep-but-appoachable analysis of a wide range of AI topics that have significant economic impact.

Ethan Mollick’s for a newsletter that lives up to its name and more.

Maggie Appleton’s Digital Garden for inspiring ideas about the design of AI.

Rex Woodbury’s for his charts that capture how the world is changing

Sebastian Raschka’s for the frequently summaries of the latest important papers in the field.

Charlie Guo’s for the easily digestable explainers of important news in AI.

Stanford’s Human-Centered Artificial Intelligence program, for their extremely informative Annual Index Report, among their many other research initiatives.

The Metaculus community, whose forecasters appear to be more accurate than others, at least as of early 2023.

19

Another prompting strategy to make an AI response sound more rich and less general has been termed “chain of density”, explained in the image below. You can try out this prompt yourself by following the link here. I wish I had come across this sooner!

23

To give you a sneak peak, here’s a distribution chart of the BLS’ forecasted annual growth rate for all occupations from 2022 to 2032, at the same scale as the distribution charts I published in my analysis of historic occupational change:

You’ll notice this projection is much more conservative than any historic change on record, especially for a decade where the nature of work could change so much. I plan to propose some areas where the BLS could improve their projections.

This chart was generated entirely by ChatGPT’s Advanced Data Analysis tool, though the complexity of the ask meant it did take me a couple of hours to get it right. This is still somewhat faster than what I would have otherwise done in Google sheets. The biggest value add I think will be helping me publish a series of interactive charts like this one, involving D3.js code that I am not familiar with— a very personal example for me of how AI tools can help beginners level up their skills more than experts.

Some of you might have found last week’s base case prediction a little boring. With the possibility of transformational AI right around the corner, why would it take 100 years to get around to only cutting our working hours in half?

Indeed, the base model was a very simple one that estimated 3% labor productivity growth from now until 2040, followed by a return to normal 1.5% growth afterwards. However, community predictions of future productivity growth on Metaculus actually appear to accelerate over time. 25% of forecasters have effectively suggested over 7.3% sustained growth from 2052 through 2122! On the other hand, even these aggressive forecasters only top out at 2.8% growth in the 2020s.

The theory behind these rapid enhancements in productivity is explained a bit in another Metaculus question that asks “If human-level artificial intelligence is developed, will World GDP grow by at least 30.0% in any of the subsequent 15 years?” The question references a theory suggested bythat productivity and GDP growth might dramatically accelerate around the point where machine intelligence begins to overtake that of humans.

Hanson’s 2001 paper, Economic Growth Given Machine Intelligence, has a few key points we should take a critical look at:

1. By 2025, give or take a few years, the cost of computing power should have continued to fall such that affordable computing hardware would be similar in power to a human brain.

2. Once machines surpass human abilities, there will be an abrupt shift from machines complementing human labor to machines replacing human labor.

3. Once this shift occurs, the constraint of a limited pool of human labor will be removed, and the quantity of labor (now provided by machine intelligence) will increase at the rate of rapidly falling computer prices (halving every two years).

4. This quick scaling of machine intelligence as a labor input could cause a 45% jump in GDP growth within a year.

What does “human-level intelligence” mean?

Point #1 is very à propos for our present moment. Computing power prices have continued to fall at the rates Hanson assumed they would back in 2001. There is however considerable uncertainty about the raw processing power of a human brain at this point.

Current Metaculus forecasts suggest that intellectual parity between machines and humans is 90% likely before 2040. There are a few other forecasts of milestones along the way, such as an LLM with approximately human-level output (between now and early 2026)1 , a “weakly general” AI that can reliably explain its reasoning in answering SAT questions and play a novel video game (between mid-2024 and 2030), and finally a the release of a general AI system as proposed by that can pass a 2-hour adversarial Turing test and have the mechanical components that would allow it to physically assemble a model (between 2026 and 2045).

The shift from human to machine

If the more aggressive timelines for this turn out to be correct, Hanson’s first hypothesis will not be far off the mark. But what about his second one regarding “an abrupt shift from machines complementing human labor to machines replacing human labor”?

First, it should be acknowledged that every job differs in its complexity and our level of trust that a new technology could do it safely and effectively. Hanson does provide a way to model this complexity to some degree. According to his model, if a job at the 75th percentile of complexity is 3 times as complex as one at the 25th percentile, it would take up to 4 years for a machine intelligence to take those middle 50% of jobs rather than some sort of immediate shift.

A 12.5% drop in employment in a single year is still wild. Such a decline has happened in individual occupations before: according to the Census, each decade has typically seen such a large, sustained drop among a handful of occupations: telemarketers and motion picture projectionists in the 2010s, textile workers in the 2000s, and statistical assistants in the 1990s.

That said, there has never been a collapse anywhere near this scale economy-wide. Even in a scenario where AI takes off more quickly than expected, I think there are a few factors that would keep the job market as a whole from imploding that quickly:

• Excepting the spring of 2020, a general 1%+ rise in unemployment in a single month hasn’t happened since December 1953. If jobs were not shed so quickly even in other recent recessions, including in 2008, it seems unlikely for the adoption of human-replacing AI to proceed at that pace.

• The various Turing tests described in the Metaculus markets might not be enough to prove that AI systems could be reliable in all of the edge cases that come up in actual jobs.

• For jobs that require physical presence, there would be additional manufacturing costs on top on innovations in AI algorithms and computing power. The production of robots can only scale up so quickly, even if they were in theory capable of doing many sorts of jobs.

• Some jobs may always require a human touch for reasons of alignment, regulation, and personal/market preference.

With this in mind we can look at how Hanson’s hypothesized “abrupt shift” affects three primary categories of jobs:

• Jobs that can be done remotely (about 30% of the workforce): potentially impacted by transformative AI sooner.

• Jobs that require physical presence, but could be automated (about 45% of the workforce): will see more direct impact from AI only after more sophisticated tests in the physical world are passed.

• Job unlikely to be replaced by machines (25% of the workforce): people will always want to work to some degree, and someone is going to have to keep the machines in line.

We can also imagine the pace of transformative AI compared to how quickly earlier transformative technologies were broadly adopted:

• Peak adoption rate of small, immediately useful technologies is about 8% of the population per year: e.g. smartphones in the 2010s, microwaves in the 1980s, radio circa 1930.

• Peak adoption rate of medium-sized tech with some learning curve is about 4% per year: e.g. PCs in the 1990s.

• Peak adoption rate of technologies with sizeable infrastructural and implementation costs is about 2-3% per year: e.g. electricity in industry between 1900-1940 or landline telephones between 1940 and 1970.

What curve might adoption of labor-replacing AI tools in virtual workplaces most closely resemble? And how differently might this play out with robots in physical workplaces?

• For virtual workplaces, there is a case for an adoption curve even faster than smartphones. End users don’t even need to purchase any physical equipment, and ChatGPT’s user base grew past 100 million faster than any other social network or tool. On the other hand, only 35% of its users find it to be anything more than “somewhat useful”. Like PCs in the 90s, there are issues with reliability, and also a learning curve to be able to make the most productive use of it. It may take time for generative AI tools to prove their reliability and fully integrate themselves with existing workflows. A good middle-of-the-road estimate might mean that the labor-replacing functions of AI might proceed overall at a similar rate as smartphone adoption, 8% per year.

• For physical work, the adoption would likely be slower, as the complexity of introducing robots into the “real” world would probably be somewhere in between the difficulty of electrification and computerization. Let’s say that labor-replacing adoption totals 3% per year once a general AI system of this capability is released.

So, if the more aggressive predictions are true, we could start seeing a significant shift in the automation of virtual work as early as next year (2024). An 8% yearly absolute rate of technological unemployment for 30% of the workforce might only be countered by the historic re-employment rate of 0.5%, so we might expect unemployment to increase (or working hours to drop) an additional 2.25% per year overall.

In May 2023, 4,000 people were estimated to have lost their jobs directly because of AI. And more broadly, employment in the information sector might be a canary in the coal mine for this trend. Q1 of 2023 saw a 0.93% drop, or 3.7% on an annualized basis. If this projection is accurate, we’d be looking at a decline about twice as strong as the Q1 2023 tech sector layoffs throughout the broader virtual labor market.

This would pick up starting in 2026 if a general AI capable of doing physical work is released by then. A 3% decline minus 0.5% re-employment across the other 45% of physical jobs with automation potential would result in a further 1.125% decline in employment each year. Based on these linear2 trends, by 2030, we’d see employment reduced by a total of 18%, or down to 33.3 hours per week if remaining work hours were distributed equitably amongst the (slightly smaller due to retirements) labor force. Could we make 4-day workweeks a norm by then?

At this point, nearly half of the work that can be done virtually has been taken over by AI. Assuming the same trend continues, we’d expect most of the remaining virtual work to be automated by 2037, even after taking a modest rate of re-employment into new jobs into account. With the continued decline in physical work as well, we’d either cross 20 hour weeks or 50% unemployment in the mid-2040s. If hours were equitably shared, we’d hit 15 hour weeks only 28 years later than Keynes predicted, in 2058. Better late than never, right?

It’s an open question where we might go from here. Barring any apocalyptic scenarios, perhaps the decline in work would level off as we get to the point where the only jobs that are left are the ones that we might never want to have machines do fully (25% in this model, but could be higher). And past a certain point, reducing the amount of work needed might not provide us much value anymore.

How much will AI continue to scale up after disrupting employment?

Speaking of value, we still need to address Hanson’s 3rd point. He assumes that once we cross the point of machines replacing rather than complementing human labor, that machine intelligence could continue scaling up at the rate at which computing power continues to fall, implying a potential 41% rise in these new “labor inputs” each year.

This might be true in some areas. As I run spreadsheets full of arrayformulas with many complex equations, I am benefiting from the computing power that used to take teams of statistical assistants and “human calculators”. Gaming in virtual reality could get increasingly vivid over time, and AI is already set to progress from generating images and short videos to full-length movies. But how much longer can the additional computing power provide us with more value, and at what point do diminishing returns limit further growth?

The limits to GDP growth

Hanson’s 4th point also makes a bold claim about GDP growth. But how the improvements I mentioned above would get reflected in GDP is debatable. While Google sheets might give me the value of a team of 1960s-era statistical assistants, what it produces doesn’t have anywhere near the same weight as the contributions of the people who used to do that work. Quality improvements, like those we might see in art, games, and videos would also be difficult for GDP to capture objectively.

For these reasons, I am hesitant to share an estimate for GDP growth along with this aggressive forecast of technological unemployment. The categories I used to group occupations is crude enough as it is. Measuring growth across each sector of the economy, and how AI might help satisfy other unmet needs would require a much more detailed approach which deserves its own post later.

I will put forward one quick hypothesis about future GDP growth for now though. There is a well-established logarithmic relationship between GDP per capita and life satisfaction (as measured by the World Happiness Report): as GDP increases, so does satisfaction, but there are significant diminishing returns.

However, the relationship between GDP per hour worked and satisfaction is much closer to linear and a tighter fit.

It is still logarithmic of course, bounded by the maximum satisfaction being 10 on a 10 point scale (and realistically somewhat lower than that). But following that trend, we’d expect to hit a collective 9 at $470/hour, which seems much more achievable than the nearly $900k we’d need to achieve a similar level of satisfaction at the same number of working hours.

$470/hour somewhat less that the 75th percentile (high) forecast for 2052, so perhaps we might arrive at both an ideal workweek and ideal level of income sometime in the 2050s?

Note: GDP per hour is invariably higher than a median hourly wage. Inequality and income from capital is likely to continue to exist even in a protopia, but a GDP per hour of $470 might imply a median hourly wage of $235/hour3 for the remaining work that is done. If you made $235 per hour, how many hours would you ideally work?

Try asking this one at your next party, and maybe you’ll get an idea of what the length of the workweek we might ultimately settle on in the far future :)

In conclusion

I have of course made tons of optimistic assumptions here that I only have 25% confidence in at most. While my predictions might still seem too conservative for some futurists, they aren’t too far off Marius Hobbhahn’s. The rates of technological unemployment I’ve proposed here are higher than most economists would predict. But even if something like this scenario does come to pass, we still have to contend with:

• The potential for inequality to skyrocket if the bargaining power of labor collapses due to fast technological unemployment.

• Misaligned AI, or humans misusing AI which results in significant catastrophe(s). Surviving such a scenario may entail employing people in much greater numbers to keep a closer eye on potential problems.

• Other “exogenous” problems that might hold back our progress such as impacts from climate change and war.

Dealing with these challenges is of course more contingent on politics than it is on technology. As bleak as questions of politics can often be, as I wrote this, I did feel optimistic that we might arrive at some imperfect solutions as the problems become clearer with time. If the headline rate of unemployment began to rise by 0.2% per month, or 0.6% per quarter, it would be immediately visible, especially if concentrated in particular industries. It wouldn’t take long for such a rise to become the top news story, and immediate economic issues are ones that politicians appear to be most sensitive to, especially if they appear to be caused by a technology that Americans already feel skeptical about. The initial solutions won’t be perfect, but I do think the pace of the problem in this scenario could be “just right” to keep the pressure on— fast enough to be taken seriously, but not so fast as to be an immediate catastrophe.

Other threats posed by AI might also arrive at the same pace as technological unemployment, and hopefully could be addressed with similar thoughtful immediacy. While I’d be out of my depth to propose solutions here, I do intend to do a few hopefully helpful things over the coming months:

• First, I also want to explore the techno-pessimist case. What if transformative AI doesn’t materialize, or comes much later than we expect? What are the odds of this happening, and what does our future look like in this scenario?

• Seeing monthly changes in unemployment by industry is a helpful “canary in the coal mine”, but I’d like to go deeper. How might changes in employment in specific occupations in the monthly Current Population Survey help us identify these trends sooner and with greater precision?

• Going beyond the various Turing tests that I linked to earlier, I think Metaculus needs more questions that refer to percentage declines in employment in specific occupations considered most susceptible to automation.

What do you think? Please share any thoughts you might have in the comments!

The first chapter of our deep dive into this history explored the long, hard weeks of the late 19th century. We then took a look at the incredible interwar drop in working hours, when a 15 hour workweek by the year 2030 really did look possible. Finally, we explored postwar stagnation in the workweek through a number of lenses, including:

• Consumer preferences and worker bargaining power

• Housing, investment, labor policies, and the role of government

• Demographics

• Productivity trends

• Different generational preferences

We’ve also looked at this chart before, to compare the US change in the workweek to the global one.

As we can see, the US is somewhat of an outlier here, though not as much as Singapore or Hong Kong. Generally, it seems like as countries become wealthier, the workweek also gets shorter. If we take the relationship between GDP per capita and working hours, weighted by population in 2019, we get this:

For every $5,000 increase in GDP per capita (in 2017 dollars), the workweek1 also gets about an hour shorter. This fairly strong relationship is also persistent over time!

We can even write an equation to calculate what the workweek might look like for a given country at a given starting GDP and productivity plus labor force participation growth rate, assuming that this relationship continues to be as durable in the future as it was in the past for most countries. We just need some solid estimates of what future growth rates might be.

For labor productivity growth, a recent study from Goldman Sachs does just that. It takes a detailed look at how emerging AI tools might complement or automate tasks of each occupation, and estimates about a 3% yearly increase in labor productivity over the next decade. This is about double our current rates, but similar to the progress seen during the electrification and post World War II economic booms:

For labor force participation growth, the BLS forecasts a 0.3% annual decline this decade, and other projections suggest a milder 0.1% annual decline during the 2030s as the remaining Baby Boomers retire.

Supposing we see 3% labor productivity growth from 2023 through 2040, the above-mentioned rates for labor force participation growth, and then return to historic average percentage changes after that, what might the workweek look like over time?

Let’s plug all of the relevant numbers into our earlier equation, using 2022 dollars instead of 2017 ones (i.e. this means it takes $6,000 of GDP increase to get a one-hour drop in the workweek). Our starting GDP is $76,630 and our starting usual hours worked per week is 39.62:

A 3% increase in labor productivity and a 0.3% decrease in labor force participation means a theoretical increase in per capita GDP of 2.7%. However, since this equation also helps us understand how much of this increase might come as a boost to GDP vs. a boost to free time, resolving it, we see that this 2.7% theoretical GDP growth is split between 2% actual GDP growth and a 0.7% decrease in working hours (this model assumes that labor productivity remains at 3% regardless of how much additional GDP is generated or how many fewer hours are worked).

For 2030, this equation projects we’ll see a 2.2 hour drop in the workweek, and another 3.1 hours shorter by 2040. After 2040, we get a surprisingly linear3 1.8 hour per week drop in the usual workweek each decade, down to 20 hours by 2120.

20 hour weeks by 2120! That has a nice ring to it. But there are a lot of understandable objections. I’ll explore an link to each of these possible refinements to the base prediction of 20 hours by 2120 as I publish them:

• What if AI is even more transformative than this scenario suggests? Barring catastrophe, how quickly would productivity and GDP improve?

• What if AI shows up everywhere but in the productivity statistics? How could new threats posed by AI, diminishing returns, and the effects of climate change keep us working long hours?

• What is the impact of a shorter workweek on satisfaction? Is 15 hours a week an “ideal” amount of work, as Keynes speculated? What would happen if we leaned further into more time off rather than more economic growth? Alternatively, if we continued working longer hours with more output, what would we consume more of?

In the meantime, if you enjoyed the GIF chart of the relationship between GDP and working hours, I have some more of this content below for 2120 Insights paid subscribers only!

Read more

Unfortunately, it wasn’t that easy. There were a few aspects of this work whose difficulty I underestimated.

First, I just needed a break! I keep track of the hours that I spent working on this project, and after a few ~20 hour weeks of this (on top of working full time), I was feeling a bit burnt out.

Taking in the sights of early spring in Chicago was restorative:

Second, I was surprised by how much the number of questions/tags in my database continued to grow. I still have way more tags than tabs.

In retrospect, this makes sense, as some of the most insightful articles I come across weave together some possible answers to many different questions.

As I added more of these tags, I also had to be thoughtful about their hierarchy. After reading more, I had to update the ones I use to keeping track of different advances in medical technology. Ultimately, good research may intersect with sub-questions in a number of these larger groups.

Even more complex than coming up with categorizations for technologies is coming up with categories for social norms, philosophies, and other important social institutions. The ambiguities of where these concepts fit in I think is reflective of the difficulties we have at times in writing and conversing about some of these topics.

Keeping a close eye on how these change over time is central to predicting the future, but they are also hard to measure, making it challenging to do any utilitarian calculus about how much satisfaction or dissatisfaction changes in these might cause.

Lastly, some subjects are just plain interesting. The perennial question of “do we have free will” might best be described as a question about what the distribution of personal agency actually is.

Wealth inequality might be the most straightforward factor that shapes that distribution, but it’s far from the only one. Wealthy individuals can only make so many decisions directly by themselves. How much does the level of trust across society influence how much agency people have independent of their level of financial wealth? And this article made me wonder, how does AI scramble that equation? Most AI disaster scenarios assume agentic AIs of the future have direct access to a significant amount of money and resources. Do we have a way of measuring any increases in their access over time?

Returning to the future of the workweek, I also thought a lot about how much work time is spent doing other activities. This is of course a difficult question to survey. How many people, when asked “OK, how much time do you actually spend working” might not fully trust who was asking the question? Would they give them a rosier answer than the truth? Indeed, there is a predictable gap between the hours that workers claim to spend on work-related activities vs. what their HR departments think.

But finding good quality data to even get the link above was a challenge! Whether I used traditional Google or fancier AI tools, there was always a layer of results that either had not cited their sources or used very crude surveys to arrive at their questionable findings. Some of the best data sources I came across were sadly out of date: salary.com did a series of surveys between 2008 and 2014 but there have been no updates since. I hope to write about this in more detail in the future, synthesizing these older surveys with new ones that focus more on flexibility and remote work.

For now, I’ve designed a few new interfaces to better keep track of what I read and how it ties back to the bigger picture.

Building a better news feed

I look back to where I was a few years ago, before I started using Twitter. At that time I was much more in the dark about the details of emerging science and technology, and the nuances of economic data that might be most useful for predicting the future. Carefully building my follow list based on the research I am doing here was great way to hack the addictiveness of social media to my benefit. It gives me a good broad sweep of information, with the downside of being the source of my many still-to-be-filed-away tabs.

But what if I want to focus on a particular area and block out the noise? Could I create a "quieter” feed for myself using AppSheet?

I’m happy to report I was successfully able to build this! Here’s what it looks like:

I’ve essentially built my own algorithm here to resurface links that I had previously saved, based on both the questions each link is associated with, and the questions I’ve identified in my list of priorities. I’m able to highlight the importance of some links over others, and might make some other improvements later. But I am very excited to have these links and notes come up automatically based on whatever I’ve decided is important at the time.

A fresh look at priorities and expectations

Reading this piece on building a PARA system changed my thinking somewhat. (thanks again !)

I had developed a good system for managing projects and short-term priorities. But there are longer-terms areas of interest too, like learning skills, or finally closing out all of these damn tabs! I ended up breaking out two separate views for these in my app. But I really manage these most closely in the source spreadsheet:

This is very helpful for setting the right expectations, which I’ve struggled with. As I mentioned back in January, some of my earlier pieces took an incredibly long time which I only realized in retrospect when I went back and tagged them with their many related questions. The articles I have planned are much more targeted, which should in theory take less time.

As you might figure from all the numbers above, I’ve come up with a way to model the estimated difficulty of each piece, starting with the number of related questions. By default, these are multiplied by my best guess of how much time it should take to research each question (estimated depth of analysis), plus the overhead time it typically takes to write anything (~2 hours). I then divide by the amount of time I think I’ll have available per week while writing the piece to get the estimated timeline. These will update automatically if I change the number of questions, depth of analysis, or the estimated time per week I think I’ll have available.

The areas of focus compete with writing time, and this system takes that into account. I even have a chart in another tab which takes a look at the time I’ve currently committed myself to, everything included. I expect to slowly chip away at the ~600 remaining tabs I have open, finishing perhaps around mid-September while I work on other stuff.

I can also use this view to see how much research I’ve already done related to the questions (the squiggly blue area). This pre-saved research might make the writing easier, but it could also mean it’s a topic with a lot of complexity. With these expectations set, it will be interesting to see how close I get to them. It’s obviously a bit of extra work to do this, but having realistic expectations helps keep this effort sustainable. This visual is one of several designed to help me recognize the real progress I make each week despite the seemingly infinite goal.

Another one of my visuals is a treemap. This has been enhanced from the last time I shared it— it now also tracks globally how much research I’ve done in some areas vs others. More research = green.

While I haven’t won the war on tabs yet, and there’s still lots of improvements to make to my knowledge management system, I’m excited to be well on the way, and to be able to keep track of multiple priorities simultaneously.

See you in 3 weeks for the return to our regular programming: my initial predictions about where the workweek might be in the year 2120!

In any normal research project, one might have a dozen or two tags that would be pretty self-contained. But my interest in writing about the future is very broad and long: there’s enough material to keep me busy for more than a lifetime. The hundreds of tabs I had open a few months ago were on their way to becoming thousands. With this level of volume, I’d either have tags that were too broad, or just too many to keep track of. I’m happy to report that after two months, I’ve finally found a happy medium by building out a hierarchy for my tags using the following principles:

1. Stay focused on the goal

2. Think like a scientist

3. Consistent language helps avoid duplicates

4. Standardize rules for each level of the hierarchy

5. Keep the hierarchy as shallow as possible

1. Stay focused on the goal

In writing 2120 Insights, my goal is to make predictions about the long-run future that are as thoughtful, optimistic, and hopefully accurate as I can. I’m not in the business of predicting specific near-term events (e.g. “Who will win the 2024 election?” or “Will there be a war between the U.S. and China before 2040?”), but I am interested in what daily life, as measured by time use surveys, might look like in the year 2120.

With this in mind, my first inclination for a tagging system was to have all of my questions and concepts related to different time use categories. While there are over 600 of these in the American Time Use Survey, I was able to settle on 6 broad primary categories and 7 secondary ones as my high-level conceptual tags.

Of course, not every important concept is directly related to how we use time. Our core motivations, level of wealth, and the trajectory of different forms of technological development are just a few of many different predictive factors.

Figuring out how to categorize these forces, and the effect they might have on daily life, was the next part of the process.

2. Think like a scientist

I did a lot of process mapping as I was thinking about what things have the biggest impact on how we end up spending time. After many iterations, I ended up with this one, which I think captures everything at a high level.

In thinking about how much time we might ideally choose to spend working vs. on other activities, the relationship between time use and all of the other concepts related to money and value (production/consumption, wealth, income) is at the center.

The environment, the natural resources available to us, and our own psychology and motivations are important influencing factors.

Finally, investing our time and wealth into better technology, social norms, and more advanced forms of government ideally helps us realize more satisfying uses of our time (and more of it as life expectancy improves).

With these as my high-level concepts, I’m able to tag what I read (or what I plan to write about) as related to either one of these concepts, OR a causal relationship between two concepts. And it’s this causality that drives predictions. For example, new tech or policy influences our productivity, which then influences our income, which ultimately influences how we use time. It’s helpful to keep my research organized along each step of this causal loop.

3. Consistent language helps avoid duplicates

If these ten high-level concepts and the relationships between them were enough to cover everything I wanted to write about, we could stop here. But of course there are additional levels of detail for each one, and the more I put in my database, the higher the risk of duplication.

This is probably the most persistently tough part of the whole organizing process, and I still struggle with it. With roughly 500+ tags and counting2 in my database, it can easily get out of hand.

Having a high level view into how I envision causality3 helps eliminate near-duplicates like:

How does labor force participation influence consumer spending/economic growth?

How does GDP influence the distribution of working hours?

Using broad terminology that includes common high-level elements either in parentheses or in the “notes” section helps me find tags more readily before I create a dupe:

Instead of: How does income inequality influence wealth and capital growth?

Use: How does income inequality (Gini index) influence wealth and capital growth?

Finally, I try to use language that’s as simple and standard as possible to phrase the questions that make up my more detailed tags, like:

How is time use is measured and predicted?

These sorts of questions help me keep track of explanations for different methodologies (which admittedly befuddled me when I first started this project).

This leads straight into my next point: