The Growth Framework IG Follow
Mind

What research actually shows about AI at work

Claims about AI range from "it will do everything" to "it is useless". Controlled studies give a more useful picture: large gains on some tasks, real mistakes on others, and the biggest help for people who are still learning.

· 7 min read · By Waseem Raja

The question

If you are deciding whether to learn AI tools for your work or freelancing, the useful question is not whether AI is impressive. It is: on which tasks does it measurably help, and where does it cause problems? A few well-designed studies help answer that.

Study 1: writing tasks

Economists Shakked Noy and Whitney Zhang ran an experiment with about 450 college-educated professionals doing realistic writing tasks such as press releases and short reports. Published in Science in 2023, it found that those with access to ChatGPT finished about 40% faster, and independent evaluators rated their work about 18% higher in quality.

What it suggests: for routine professional writing, AI can save a lot of time. Limits: the tasks were short and set in an experiment, not months of real work.

Study 2: customer support

Erik Brynjolfsson, Danielle Li and Lindsey Raymond studied more than 5,000 customer support agents at a software company as an AI assistant was introduced. On average, agents resolved about 14% more issues per hour. The gains were largest, around 34%, for newer and less-skilled agents, while the most experienced agents gained little.

What it suggests: AI can help newcomers learn and perform faster. For people early in their careers, that is significant.

Study 3: consultants and the "jagged frontier"

Researchers from Harvard, MIT, Wharton and elsewhere worked with Boston Consulting Group on an experiment with 758 consultants. For tasks within AI's capabilities, consultants using GPT-4 completed about 12% more tasks, about 25% faster, with work rated over 40% higher in quality.

But for a task chosen to be just outside AI's capabilities, consultants using AI were about 19 percentage points less likely to reach the correct answer than those without it. The AI gave confident but wrong analysis, and many people trusted it.

What it suggests: AI's abilities are uneven, which the researchers called a "jagged frontier". The danger is not only that AI makes mistakes, but that its mistakes look convincing.

Study 4: programming

In a controlled experiment by researchers including GitHub staff, developers using GitHub Copilot completed a set programming task about 55% faster than those without it. Limits: it was one task, and speed alone does not measure code quality or long-term maintenance.

What these studies agree on

  • Gains are real but task-specific, not universal.
  • Beginners often benefit most, because AI fills gaps in experience.
  • Checking matters. On unfamiliar or tricky tasks, uncritical use can make results worse.

What remains uncertain

  • How results from short experiments translate to years of real work.
  • Effects on wages and job availability, which are still being studied and debated.
  • How quickly newer tools change these numbers; some results may already be out of date.

What this means in Pakistan

Pakistan has a large freelance workforce in writing, design, support and software. The research suggests AI tools can make that work faster, and help newer freelancers compete. It also suggests clients will expect more output for the same price over time, so the advantage lies in combining AI speed with skill, judgement and checking.

Practical takeaway

  1. Use AI first for drafting, summarising and routine writing, where evidence of gains is strongest.
  2. For facts, numbers, legal, medical or financial claims, verify everything.
  3. Keep a simple log for two weeks: tasks where AI helped, and tasks where it caused errors. Your own "frontier" map is more useful than general claims.

Key idea

AI is a strong assistant on some tasks and a confident guesser on others. Your job is to know which is which.

Figures are as reported by the cited studies, rounded. Study conditions vary and results may not apply to every job.

Sources

  1. Noy S, Zhang W (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science
  2. Brynjolfsson E, Li D, Raymond LR (2023). Generative AI at work. NBER Working Paper 31161
  3. Dell'Acqua F et al. (2023). Navigating the jagged technological frontier: field experimental evidence of the effects of AI on knowledge worker productivity and quality. Harvard Business School Working Paper 24-013
  4. Peng S et al. (2023). The impact of AI on developer productivity: evidence from GitHub Copilot. arXiv:2302.06590

Educational content only. This is not medical or financial advice. Check facts with qualified professionals before you act.