Corporate America has unmistakably drifted leftward over the past decade, leaving behind a trail of racial-equity statements and climate promises. A lively new academic working paper, written by finance professors William M. Cassidy and Elisabeth Kempf and released through the National Bureau of Economic Research, quantifies the change based on tweets by S&P 500 companies—and explores both the causes and the consequences of this shift.

To measure corporate partisanship, the authors statistically compare the language used in tweets of companies with that used in tweets by Democratic and Republican members of Congress. Starting in late 2017 and accelerating in 2019, the once-rare phenomenon of partisan corporate tweets picked up. The shift was driven primarily by Democratic-leaning language about topics like DEI and climate change, though Republican-leaning language also became more common.

Those results are about what a casual observer would expect, though the exact timing is interesting. The 2017–2019 period begins the year after Donald Trump’s election and in between the Black Lives Matter moments of 2015 and 2020, rather than directly coinciding with a major national political development.

The authors report that “the growth of Democratic-leaning corporate speech is closely correlated with the expansion of assets under management in funds with environmental, social, and governance (ESG) objectives.” Firms with high BlackRock ownership also saw a particularly large increase after 2019, when CEO Larry Fink sent a letter encouraging corporations to dive into political controversies.

In other words, companies largely appear to be responding to direct pressure and financial incentives, as opposed to reacting to broader political developments and cultural phenomena. In addition to the overall upward trend, though, some abrupt, temporary spikes did occur around events such as the death of George Floyd in May 2020.

However, in the wake of a partisan tweet, a company’s stock price tended to decline about 0.3 percent over the next 13 days before leveling off. This effect was less severe when the partisan nature of the tweet aligned with the preferences of the industry’s investors and workers.

How does this all fit together?

The authors propose a theoretical model in which Democrats’ ESG push forced corporations to pick sides, necessarily alienating investors of one political persuasion or the other. This could explain the apparent paradox in which corporations responded to financial incentives and somehow hurt their stock. Taking the other side might have hurt their stock even more, and even staying out of the fray could have cost a company the support of ESG groups over the long run. Of course, it’s also possible that some corporations simply misjudged the tradeoffs they faced, lacked the backbone to resist pressure, or drifted left out of genuine conviction.

The study’s data end in early 2023, so the authors “cannot examine whether and how corporate speech patterns have responded to increasing political backlash, especially against ESG and DEI initiatives, as well as broader shifts in the political climate.” It remains to be seen how corporations behave in the Trump II era. Will they depoliticize, cater to the administration’s whims to the chagrin of Democratic and ESG investors—or just stay woke?

Photo by Erik McGregor/LightRocket via Getty Images

• Focus: The study explores the potential of residential air conditioners for providing frequency regulation to the electric power grid.

• Method: The research uses ON/OFF mode adjustments of air conditioners for aggregate power control.

• Scope: The approach accommodates both single and multi-zone houses.

• Challenges: Real-world experiments revealed practical issues and the authors discuss hardware/software solutions.

• Impediment: Current thermostat API limitations pose significant challenges to fast load control applications.

• Origin: The article explores the invention of the wheel, discussing its potential origins and evolution.

• Theory: It highlights a theory proposing the wheel's invention by miners in the Carpathian Mountains around 3900 B.C.E. to transport copper ore.

• Evidence: The theory is supported by the discovery of miniature clay wagons in the region, suggesting early wheeled transport.

• Evolution: The article discusses the transition from rollers to wheels, emphasizing the role of mechanical advantage and simulations in this process.

• Conclusion: The author suggests that the wheel's invention was a gradual process, with continuous improvements leading to its current form.

How and When Was the Wheel Invented?

By Kai James
June 12, 2025

Jon Cartagena

X

Story Stream

recent articles

• Video: Global Warming Lorem Ipsum Dolor Sit ...
• Article: Global Warming Lorem Ipsum Dolor Sit ...
• Article: Global Warming Lorem Ipsum Dolor Sit ...
• Entry: Global Warming Lorem Ipsum Dolor Sit ...
• Video: Global Warming Lorem Ipsum Dolor Sit ...

Imagine you’re a copper miner in southeastern Europe in the year 3900 B.C.E. Day after day you haul copper ore through the mine’s sweltering tunnels.

You’ve resigned yourself to the grueling monotony of mining life. Then one afternoon, you witness a fellow worker doing something remarkable.

With an odd-looking contraption, he casually transports the equivalent of three times his body weight on a single trip. As he returns to the mine to fetch another load, it suddenly dawns on you that your chosen profession is about to get far less taxing and much more lucrative.

What you don’t realize: You’re witnessing something that will change the course of history – not just for your tiny mining community, but for all of humanity.

Despite the wheel’s immeasurable impact, no one is certain as to who invented it, or when and where it was first conceived. The hypothetical scenario described above is based on a 2015 theory that miners in the Carpathian Mountains – in present-day Hungary – first invented the wheel nearly 6,000 years ago as a means to transport copper ore.

The theory is supported by the discovery of more than 150 miniaturized wagons by archaeologists working in the region. These pint-sized, four-wheeled models were made from clay, and their outer surfaces were engraved with a wickerwork pattern reminiscent of the basketry used by mining communities at the time. Carbon dating later revealed that these wagons are the earliest known depictions of wheeled transport to date.

This theory also raises a question of particular interest to me, an aerospace engineer who studies the science of engineering design. How did an obscure, scientifically naive mining society discover the wheel, when highly advanced civilizations, such as the ancient Egyptians, did not?

A controversial idea

It has long been assumed that wheels evolved from simple wooden rollers. But until recently no one could explain how or why this transformation took place. What’s more, beginning in the 1960s, some researchers started to express strong doubts about the roller-to-wheel theory.

After all, for rollers to be useful, they require flat, firm terrain and a path free of inclines and sharp curves. Furthermore, once the cart passes them, used rollers need to be continually brought around to the front of the line to keep the cargo moving. For all these reasons, the ancient world used rollers sparingly. According to the skeptics, rollers were too rare and too impractical to have been the starting point for the evolution of the wheel.

But a mine – with its enclosed, human-made passageways – would have provided favorable conditions for rollers. This factor, among others, compelled my team to revisit the roller hypothesis.

A turning point

The transition from rollers to wheels requires two key innovations. The first is a modification of the cart that carries the cargo. The cart’s base must be outfitted with semicircular sockets, which hold the rollers in place. This way, as the operator pulls the cart, the rollers are pulled along with it.

This innovation may have been motivated by the confined nature of the mine environment, where having to periodically carry used rollers back around to the front of the cart would have been especially onerous.

The discovery of socketed rollers represented a turning point in the evolution of the wheel and paved the way for the second and most important innovation. This next step involved a change to the rollers themselves. To understand how and why this change occurred, we turned to physics and computer-aided engineering.

Simulating the wheel’s evolution

To begin our investigation, we created a computer program designed to simulate the evolution from a roller to a wheel. Our hypothesis was that this transformation was driven by a phenomenon called “mechanical advantage.” This same principle allows pliers to amplify a user’s grip strength by providing added leverage. Similarly, if we could modify the shape of the roller to generate mechanical advantage, this would amplify the user’s pushing force, making it easier to advance the cart.

Our algorithm worked by modeling hundreds of potential roller shapes and evaluating how each one performed, both in terms of mechanical advantage and structural strength. The latter was used to determine whether a given roller would break under the weight of the cargo. As predicted, the algorithm ultimately converged upon the familiar wheel-and-axle shape, which it determined to be optimal.

During the execution of the algorithm, each new design performed slightly better than its predecessor. We believe a similar evolutionary process played out with the miners 6,000 years ago.

It is unclear what initially prompted the miners to explore alternative roller shapes. One possibility is that friction at the roller-socket interface caused the surrounding wood to wear away, leading to a slight narrowing of the roller at the point of contact. Another theory is that the miners began thinning out the rollers so that their carts could pass over small obstructions on the ground.

Either way, thanks to mechanical advantage, this narrowing of the axle region made the carts easier to push. As time passed, better-performing designs were repeatedly favored over the others, and new rollers were crafted to mimic these top performers.

Consequently, the rollers became more and more narrow, until all that remained was a slender bar capped on both ends by large discs. This rudimentary structure marks the birth of what we now refer to as “the wheel.”

According to our theory, there was no precise moment at which the wheel was invented. Rather, just like the evolution of species, the wheel emerged gradually from an accumulation of small improvements.

This is just one of the many chapters in the wheel’s long and ongoing evolution. More than 5,000 years after the contributions of the Carpathian miners, a Parisian bicycle mechanic invented radial ball bearings, which once again revolutionized wheeled transportation.

Ironically, ball bearings are conceptually identical to rollers, the wheel’s evolutionary precursor. Ball bearings form a ring around the axle, creating a rolling interface between the axle and the wheel hub, thereby circumventing friction. With this innovation, the evolution of the wheel came full circle.

This example also shows how the wheel’s evolution, much like its iconic shape, traces a circuitous path – one with no clear beginning, no end, and countless quiet revolutions along the way.

Kai James, Professor of Aerospace Engineering, Georgia Institute of Technology

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Recommended Newsletters

Carl M. Cannon's Morning Note

headlines against the historic events

Science News

Featuring the latest science news

Related Topics: Wheel, history, engineering

Comment

Show comments Hide Comments

You must be logged in to comment.

RCS Account: Login Register

• The article addresses the potential damage to American science caused by proposed cuts in federal research funding and a decline in public trust.

• It argues that the culture of science has shifted towards prioritizing prestige and visibility, which has weakened the connection between science and the public.

• The author cites a Pew Research Center report indicating a decrease in public trust in scientists and their communication abilities.

• The author warns that other nations, like China, are investing heavily in science, posing a potential threat to U.S. leadership in this field.

• The author calls on both Congress and scientists to reaffirm their commitment to public good and address the issues undermining public trust in science.

American Science's Culture Has Contributed to the Grave Threat It Now Faces

By Brad Schwartz, MD
June 12, 2025

AP

X

Story Stream

recent articles

• Video: Global Warming Lorem Ipsum Dolor Sit ...
• Article: Global Warming Lorem Ipsum Dolor Sit ...
• Article: Global Warming Lorem Ipsum Dolor Sit ...
• Entry: Global Warming Lorem Ipsum Dolor Sit ...
• Video: Global Warming Lorem Ipsum Dolor Sit ...

Congress is now considering significant cuts to federal research funding. These decisions, if approved, won’t just stall scientific progress in the short term—they would mark a strategic retreat from one of America’s greatest national strengths. Federally supported basic research yields the foundation for new technologies, and indeed new industries, that give the United States the vibrant economy that is the envy of the world.

The debate over these cuts has focused largely on numbers. But the larger issue runs deeper. Federal support for science depends on public trust. And that trust, shaped over decades, is showing signs of real strain.

This is not just a political story; it’s also a cultural one with many contributors. Since I’m a scientist, I’ll address science’s contribution to our current situation.

Over time, the culture of science has become increasingly focused on metrics of prestige: grant dollars, publication volume, high-profile coverage. We’ve come to equate visibility with value. In subtle but important ways, science has absorbed the habits of celebrity—chasing recognition, measuring performance by attention, and celebrating status over service.

None of this started with bad intentions, but it has consequences. Most importantly, it has weakened the connection between science and the public.

A recent Pew Research Center report makes this shift visible. While most Americans still express some confidence in scientists, only 26 percent say they have a “great deal” of trust—down sharply from early in the pandemic. Just 45 percent think scientists are good communicators. Nearly half believe scientists “feel superior to others.” 

These aren’t just optics problems. They reflect a deeper question about whether science still appears to be accountable to the public—and whether it is seen as a shared enterprise or a professional class.

This erosion of trust makes it easier for lawmakers to treat science funding as negotiable and to treat scientific research as a political football. Fostering prestige or celebrity removes focus from the discoveries that make life better and changes the way society thinks about science. If science feels distant or self-interested, it becomes politically expendable.

Meanwhile, other nations are moving in the opposite direction. China, in particular, continues to treat science as a national strategic priority. It is investing in research infrastructure, recruiting talent, and adapting many of the same principles that made U.S. science globally dominant in the postwar era. There’s urgency in that investment, and a clear message: discovery and innovation are not side interests—they are central to national leadership.

Around the time of the last U.S. presidential election, Chinese scientists were reportedly sharing a pointed phrase online: Now is our time. They saw an opening—an expectation that the United States might retreat from bold, curiosity-driven research. They were preparing to step in.

That possibility is no longer hypothetical. According to GlobalData, China has surpassed the United States in the annual number of clinical trials.

As a nation, we need to act quickly—and with clarity.

To Congress, I would say: these proposed cuts are not just about budget lines. They are about national priorities. We cannot compete economically or geopolitically without investing in the science that that has made us the world leader in health, technology, security, and energy. Defunding that capacity now will send a message to the world—and to our own people—that we’ve chosen short-term savings or political infighting over long-term leadership.

To my colleagues in science, this moment demands reflection. Most scientists I know are deeply committed to discovery and public good. But we have to acknowledge how our institutions and incentives have drifted. When recognition becomes the goal, and not the byproduct, we lose sight of our mission. And when we speak mainly to each other, we risk being tuned out by everyone else.

This is not about image management. It’s about regaining trust by returning to our roots: asking hard questions, recognizing the societal importance of our role, and staying grounded in the needs and concerns of the people we serve.

After World War II, the United States built a research model based on a shared understanding: the federal government would invest in discovery, and research institutions would foster the talent and integrity to pursue it. That model worked because it was grounded in mutual responsibility and public purpose.

We don’t need to reinvent that model, but we do need to recommit to it. And we need to be honest about what that takes.

That means asking whether our systems reflect the values we claim to uphold. It means funding science not because it’s popular, but because it prepares us for the future. And it means showing the public that we know who science is really for.

Congress still has a chance to send a signal that discovery matters, that truth-seeking matters, and that American leadership in science is not negotiable.

Brad Schwartz, MD, is CEO of the Morgridge Institute for Research, a nonprofit biomedical research institute affiliated with the University of Wisconsin–Madison.

Recommended Newsletters

Carl M. Cannon's Morning Note

headlines against the historic events

Science News

Featuring the latest science news

Related Topics: science funding, academia, science, Politics

Comment

Show comments Hide Comments

You must be logged in to comment.

RCS Account: Login Register

• The article details: Amsterdam's experiment with an AI system ('Smart Check') to detect welfare fraud.

• The system aimed: To improve efficiency and remove bias, but faced challenges.

• Key issues: Concerns about fairness, bias against certain groups, and the impact on individuals were raised.

• The pilot program: Found the system flagged more applicants than caseworkers, and showed bias, leading to its termination.

• The conclusion: Raises questions about the feasibility of fair AI in welfare systems and the importance of considering ethical and political values.

This story is a partnership between MIT Technology Review, Lighthouse Reports, and Trouw, and was supported by the Pulitzer Center. 

Two futures

Hans de Zwart, a gym teacher turned digital rights advocate, says that when he saw Amsterdam’s plan to have an algorithm evaluate every welfare applicant in the city for potential fraud, he nearly fell out of his chair. 

It was February 2023, and de Zwart, who had served as the executive director of Bits of Freedom, the Netherlands’ leading digital rights NGO, had been working as an informal advisor to Amsterdam’s city government for nearly two years, reviewing and providing feedback on the AI systems it was developing. 

According to the city’s documentation, this specific AI model—referred to as “Smart Check”—would consider submissions from potential welfare recipients and determine who might have submitted an incorrect application. More than any other project that had come across his desk, this one stood out immediately, he told us—and not in a good way. “There’s some very fundamental [and] unfixable problems,” he says, in using this algorithm “on real people.”

Advertisement

From his vantage point behind the sweeping arc of glass windows at Amsterdam’s city hall, Paul de Koning, a consultant to the city whose résumé includes stops at various agencies in the Dutch welfare state, had viewed the same system with pride. De Koning, who managed Smart Check’s pilot phase, was excited about what he saw as the project’s potential to improve efficiency and remove bias from Amsterdam’s social benefits system. 

A team of fraud investigators and data scientists had spent years working on Smart Check, and de Koning believed that promising early results had vindicated their approach. The city had consulted experts, run bias tests, implemented technical safeguards, and solicited feedback from the people who’d be affected by the program—more or less following every recommendation in the ethical-AI playbook. “I got a good feeling,” he told us. 

These opposing viewpoints epitomize a global debate about whether algorithms can ever be fair when tasked with making decisions that shape people’s lives. Over the past several years of efforts to use artificial intelligence in this way, examples of collateral damage have mounted: nonwhite job applicants weeded out of job application pools in the US, families being wrongly flagged for child abuse investigations in Japan, and low-income residents being denied food subsidies in India. 

Proponents of these assessment systems argue that they can create more efficient public services by doing more with less and, in the case of welfare systems specifically, reclaim money that is allegedly being lost from the public purse. In practice, many were poorly designed from the start. They sometimes factor in personal characteristics in a way that leads to discrimination, and sometimes they have been deployed without testing for bias or effectiveness. In general, they offer few options for people to challenge—or even understand—the automated actions directly affecting how they live. 

The result has been more than a decade of scandals. In response, lawmakers, bureaucrats, and the private sector, from Amsterdam to New York, Seoul to Mexico City, have been trying to atone by creating algorithmic systems that integrate the principles of “responsible AI”—an approach that aims to guide AI development to benefit society while minimizing negative consequences. 

Developing and deploying ethical AI is a top priority for the European Union, and the same was true for the US under former president Joe Biden, who released a blueprint for an AI Bill of Rights. That plan was rescinded by the Trump administration, which has removed considerations of equity and fairness, including in technology, at the national level. Nevertheless, systems influenced by these principles are still being tested by leaders in countries, states, provinces, and cities—in and out of the US—that have immense power to make decisions like whom to hire, when to investigate cases of potential child abuse, and which residents should receive services first. 

Amsterdam indeed thought it was on the right track. City officials in the welfare department believed they could build technology that would prevent fraud while protecting citizens’ rights. They followed these emerging best practices and invested a vast amount of time and money in a project that eventually processed live welfare applications. But in their pilot, they found that the system they’d developed was still not fair and effective. Why? 

Lighthouse Reports, MIT Technology Review, and the Dutch newspaper Trouw have gained unprecedented access to the system to try to find out. In response to a public records request, the city disclosed multiple versions of the Smart Check algorithm and data on how it evaluated real-world welfare applicants, offering us unique insight into whether, under the best possible conditions, algorithmic systems can deliver on their ambitious promises.  

The answer to that question is far from simple. For de Koning, Smart Check represented technological progress toward a fairer and more transparent welfare system. For de Zwart, it represented a substantial risk to welfare recipients’ rights that no amount of technical tweaking could fix. As this algorithmic experiment unfolded over several years, it called into question the project’s central premise: that responsible AI can be more than a thought experiment or corporate selling point—and actually make algorithmic systems fair in the real world.

A chance at redemption

Understanding how Amsterdam found itself conducting a high-stakes endeavor with AI-driven fraud prevention requires going back four decades, to a national scandal around welfare investigations gone too far. 

Advertisement

In 1984, Albine Grumböck, a divorced single mother of three, had been receiving welfare for several years when she learned that one of her neighbors, an employee at the social service’s local office, had been secretly surveilling her life. He documented visits from a male friend, who in theory could have been contributing unreported income to the family. On the basis of his observations, the welfare office cut Grumböck’s benefits. She fought the decision in court and won.

Despite her personal vindication, Dutch welfare policy has continued to empower welfare fraud investigators, sometimes referred to as “toothbrush counters,” to turn over people’s lives. This has helped create an atmosphere of suspicion that leads to problems for both sides, says Marc van Hoof, a lawyer who has helped Dutch welfare recipients navigate the system for decades: “The government doesn’t trust its people, and the people don’t trust the government.”

Harry Bodaar, a career civil servant, has observed the Netherlands’ welfare policy up close throughout much of this time—first as a social worker, then as a fraud investigator, and now as a welfare policy advisor for the city. The past 30 years have shown him that “the system is held together by rubber bands and staples,” he says. “And if you’re at the bottom of that system, you’re the first to fall through the cracks.”

Making the system work better for beneficiaries, he adds, was a large motivating factor when the city began designing Smart Check in 2019. “We wanted to do a fair check only on the people we [really] thought needed to be checked,” Bodaar says—in contrast to previous department policy, which until 2007 was to conduct home visits for every applicant. 

But he also knew that the Netherlands had become something of a ground zero for problematic welfare AI deployments. The Dutch government’s attempts to modernize fraud detection through AI had backfired on a few notorious occasions.

In 2019, it was revealed that the national government had been using an algorithm to create risk profiles that it hoped would help spot fraud in the child care benefits system. The resulting scandal saw nearly 35,000 parents, most of whom were migrants or the children of migrants, wrongly accused of defrauding the assistance system over six years. It put families in debt, pushed some into poverty, and ultimately led the entire government to resign in 2021.  

In Rotterdam, a 2023 investigation by Lighthouse Reports into a system for detecting welfare fraud found it to be biased against women, parents, non-native Dutch speakers, and other vulnerable groups, eventually forcing the city to suspend use of the system. Other cities, like Amsterdam and Leiden, used a system called the Fraud Scorecard, which was first deployed more than 20 years ago and included education, neighborhood, parenthood, and gender as crude risk factors to assess welfare applicants; that program was also discontinued.

The Netherlands is not alone. In the United States, there have been at least 11 cases in which state governments used algorithms to help disperse public benefits, according to the nonprofit Benefits Tech Advocacy Hub, often with troubling results. Michigan, for instance, falsely accused 40,000 people of committing unemployment fraud. And in France, campaigners are taking the national welfare authority to court over an algorithm they claim discriminates against low-income applicants and people with disabilities. 

This string of scandals, as well as a growing awareness of how racial discrimination can be embedded in algorithmic systems, helped fuel the growing emphasis on responsible AI. It’s become “this umbrella term to say that we need to think about not just ethics, but also fairness,” says Jiahao Chen, an ethical-AI consultant who has provided auditing services to both private and local government entities. “I think we are seeing that realization that we need things like transparency and privacy, security and safety, and so on.” 

The approach, based on a set of tools intended to rein in the harms caused by the proliferating technology, has given rise to a rapidly growing field built upon a familiar formula: white papers and frameworks from think tanks and international bodies, and a lucrative consulting industry made up of traditional power players like the Big 5 consultancies, as well as a host of startups and nonprofits. In 2019, for instance, the Organisation for Economic Co-operation and Development, a global economic policy body, published its Principles on Artificial Intelligence as a guide for the development of “trustworthy AI.” Those principles include building explainable systems, consulting public stakeholders, and conducting audits. 

Advertisement

But the legacy left by decades of algorithmic misconduct has proved hard to shake off, and there is little agreement on where to draw the line between what is fair and what is not. While the Netherlands works to institute reforms shaped by responsible AI at the national level, Algorithm Audit, a Dutch NGO that has provided ethical-AI auditing services to government ministries, has concluded that the technology should be used to profile welfare recipients only under strictly defined conditions, and only if systems avoid taking into account protected characteristics like gender. Meanwhile, Amnesty International, digital rights advocates like de Zwart, and some welfare recipients themselves argue that when it comes to making decisions about people’s lives, as in the case of social services, the public sector should not be using AI at all.

Amsterdam hoped it had found the right balance. “We’ve learned from the things that happened before us,” says Bodaar, the policy advisor, of the past scandals. And this time around, the city wanted to build a system that would “show the people in Amsterdam we do good and we do fair.”

Finding a better way

Every time an Amsterdam resident applies for benefits, a caseworker reviews the application for irregularities. If an application looks suspicious, it can be sent to the city’s investigations department—which could lead to a rejection, a request to correct paperwork errors, or a recommendation that the candidate receive less money. Investigations can also happen later, once benefits have been dispersed; the outcome may force recipients to pay back funds, and even push some into debt.

Officials have broad authority over both applicants and existing welfare recipients. They can request bank records, summon beneficiaries to city hall, and in some cases make unannounced visits to a person’s home. As investigations are carried out—or paperwork errors fixed—much-needed payments may be delayed. And often—in more than half of the investigations of applications, according to figures provided by Bodaar—the city finds no evidence of wrongdoing. In those cases, this can mean that the city has “wrongly harassed people,” Bodaar says. 

The Smart Check system was designed to avoid these scenarios by eventually replacing the initial caseworker who flags which cases to send to the investigations department. The algorithm would screen the applications to identify those most likely to involve major errors, based on certain personal characteristics, and redirect those cases for further scrutiny by the enforcement team.

If all went well, the city wrote in its internal documentation, the system would improve on the performance of its human caseworkers, flagging fewer welfare applicants for investigation while identifying a greater proportion of cases with errors. In one document, the city projected that the model would prevent up to 125 individual Amsterdammers from facing debt collection and save €2.4 million annually. 

Smart Check was an exciting prospect for city officials like de Koning, who would manage the project when it was deployed. He was optimistic, since the city was taking a scientific approach, he says; it would “see if it was going to work” instead of taking the attitude that “this must work, and no matter what, we will continue this.”

It was the kind of bold idea that attracted optimistic techies like Loek Berkers, a data scientist who worked on Smart Check in only his second job out of college. Speaking in a cafe tucked behind Amsterdam’s city hall, Berkers remembers being impressed at his first contact with the system: “Especially for a project within the municipality,” he says, it “was very much a sort of innovative project that was trying something new.”

Smart Check made use of an algorithm called an “explainable boosting machine,” which allows people to more easily understand how AI models produce their predictions. Most other machine-learning models are often regarded as “black boxes” running abstract mathematical processes that are hard to understand for both the employees tasked with using them and the people affected by the results. 

The Smart Check model would consider 15 characteristics—including whether applicants had previously applied for or received benefits, the sum of their assets, and the number of addresses they had on file—to assign a risk score to each person. It purposefully avoided demographic factors, such as gender, nationality, or age, that were thought to lead to bias. It also tried to avoid “proxy” factors—like postal codes—that may not look sensitive on the surface but can become so if, for example, a postal code is statistically associated with a particular ethnic group.

In an unusual step, the city has disclosed this information and shared multiple versions of the Smart Check model with us, effectively inviting outside scrutiny into the system’s design and function. With this data, we were able to build a hypothetical welfare recipient to get insight into how an individual applicant would be evaluated by Smart Check.  

This model was trained on a data set encompassing 3,400 previous investigations of welfare recipients. The idea was that it would use the outcomes from these investigations, carried out by city employees, to figure out which factors in the initial applications were correlated with potential fraud. 

Advertisement

But using past investigations introduces potential problems from the start, says Sennay Ghebreab, scientific director of the Civic AI Lab (CAIL) at the University of Amsterdam, one of the external groups that the city says it consulted with. The problem of using historical data to build the models, he says, is that “we will end up [with] historic biases.” For example, if caseworkers historically made higher rates of mistakes with a specific ethnic group, the model could wrongly learn to predict that this ethnic group commits fraud at higher rates. 

The city decided it would rigorously audit its system to try to catch such biases against vulnerable groups. But how bias should be defined, and hence what it actually means for an algorithm to be fair, is a matter of fierce debate. Over the past decade, academics have proposed dozens of competing mathematical notions of fairness, some of which are incompatible. This means that a system designed to be “fair” according to one such standard will inevitably violate others.

Amsterdam officials adopted a definition of fairness that focused on equally distributing the burden of wrongful investigations across different demographic groups. 

Title

Subtitle

Text

In other words, they hoped this approach would ensure that welfare applicants of different backgrounds would carry the same burden of being incorrectly investigated at similar rates. 

Mixed feedback

As it built Smart Check, Amsterdam consulted various public bodies about the model, including the city’s internal data protection officer and the Amsterdam Personal Data Commission. It also consulted private organizations, including the consulting firm Deloitte. Each gave the project its approval. 

But one key group was not on board: the Participation Council, a 15-member advisory committee composed of benefits recipients, advocates, and other nongovernmental stakeholders who represent the interests of the people the system was designed to help—and to scrutinize. The committee, like de Zwart, the digital rights advocate, was deeply troubled by what the system could mean for individuals already in precarious positions. 

Anke van der Vliet, now in her 70s, is one longtime member of the council. After she sinks slowly from her walker into a seat at a restaurant in Amsterdam’s Zuid neighborhood, where she lives, she retrieves her reading glasses from their case. “We distrusted it from the start,” she says, pulling out a stack of papers she’s saved on Smart Check. “Everyone was against it.”

For decades, she has been a steadfast advocate for the city’s welfare recipients—a group that, by the end of 2024, numbered around 35,000. In the late 1970s, she helped found Women on Welfare, a group dedicated to exposing the unique challenges faced by women within the welfare system.

City employees first presented their plan to the Participation Council in the fall of 2021. Members like van der Vliet were deeply skeptical. “We wanted to know, is it to my advantage or disadvantage?” she says. 

Two more meetings could not convince them. Their feedback did lead to key changes—including reducing the number of variables the city had initially considered to calculate an applicant’s score and excluding variables that could introduce bias, such as age, from the system. But the Participation Council stopped engaging with the city’s development efforts altogether after six months. “The Council is of the opinion that such an experiment affects the fundamental rights of citizens and should be discontinued,” the group wrote in March 2022. Since only around 3% of welfare benefit applications are fraudulent, the letter continued, using the algorithm was “disproportionate.”

Advertisement

De Koning, the project manager, is skeptical that the system would ever have received the approval of van der Vliet and her colleagues. “I think it was never going to work that the whole Participation Council was going to stand behind the Smart Check idea,” he says. “There was too much emotion in that group about the whole process of the social benefit system.” He adds, “They were very scared there was going to be another scandal.” 

But for advocates working with welfare beneficiaries, and for some of the beneficiaries themselves, the worry wasn’t a scandal but the prospect of real harm. The technology could not only make damaging errors but leave them even more difficult to correct—allowing welfare officers to “hide themselves behind digital walls,” says Henk Kroon, an advocate who assists welfare beneficiaries at the Amsterdam Welfare Association, a union established in the 1970s. Such a system could make work “easy for [officials],” he says. “But for the common citizens, it’s very often the problem.” 

Time to test 

Despite the Participation Council’s ultimate objections, the city decided to push forward and put the working Smart Check model to the test. 

The first results were not what they’d hoped for. When the city’s advanced analytics team ran the initial model in May 2022, they found that the algorithm showed heavy bias against migrants and men, which we were able to independently verify. 

As the city told us and as our analysis confirmed, the initial model was more likely to wrongly flag non-Dutch applicants. And it was nearly twice as likely to wrongly flag an applicant with a non-Western nationality than one with a Western nationality. The model was also 14% more likely to wrongly flag men for investigation. 

In the process of training the model, the city also collected data on who its human case workers had flagged for investigation and which groups the wrongly flagged people were more likely to belong to. In essence, they ran a bias test on their own analog system—an important way to benchmark that is rarely done before deploying such systems. 

What they found in the process led by caseworkers was a strikingly different pattern. Whereas the Smart Check model was more likely to wrongly flag non-Dutch nationals and men, human caseworkers were more likely to wrongly flag Dutch nationals and women. 

The team behind Smart Check knew that if they couldn’t correct for bias, the project would be canceled. So they turned to a technique from academic research, known as training-data reweighting. In practice, that meant applicants with a non-Western nationality who were deemed to have made meaningful errors in their applications were given less weight in the data, while those with a Western nationality were given more.

Eventually, this appeared to solve their problem: As Lighthouse’s analysis confirms, once the model was reweighted, Dutch and non-Dutch nationals were equally likely to be wrongly flagged. 

De Koning, who joined the Smart Check team after the data was reweighted, said the results were a positive sign: “Because it was fair … we could continue the process.” 

The model also appeared to be better than caseworkers at identifying applications worthy of extra scrutiny, with internal testing showing a 20% improvement in accuracy.

Buoyed by these results, in the spring of 2023, the city was almost ready to go public. It submitted Smart Check to the Algorithm Register, a government-run transparency initiative meant to keep citizens informed about machine-learning algorithms either in development or already in use by the government.

For de Koning, the city’s extensive assessments and consultations were encouraging, particularly since they also revealed the biases in the analog system. But for de Zwart, those same processes represented a profound misunderstanding: that fairness could be engineered. 

Advertisement

In a letter to city officials, de Zwart criticized the premise of the project and, more specifically, outlined the unintended consequences that could result from reweighting the data. It might reduce bias against people with a migration background overall, but it wouldn’t guarantee fairness across intersecting identities; the model could still discriminate against women with a migration background, for instance. And even if that issue were addressed, he argued, the model might still treat migrant women in certain postal codes unfairly, and so on. And such biases would be hard to detect.

“The city has used all the tools in the responsible-AI tool kit,” de Zwart told us. “They have a bias test, a human rights assessment; [they have] taken into account automation bias—in short, everything that the responsible-AI world recommends. Nevertheless, the municipality has continued with something that is fundamentally a bad idea.”

Ultimately, he told us, it’s a question of whether it’s legitimate to use data on past behavior to judge “future behavior of your citizens that fundamentally you cannot predict.” 

Officials still pressed on—and set March 2023 as the date for the pilot to begin. Members of Amsterdam’s city council were given little warning. In fact, they were only informed the same month—to the disappointment of Elisabeth IJmker, a first-term council member from the Green Party, who balanced her role in municipal government with research on religion and values at Amsterdam’s Vrije University. 

“Reading the words ‘algorithm’ and ‘fraud prevention’ in one sentence, I think that’s worth a discussion,” she told us. But by the time that she learned about the project, the city had already been working on it for years. As far as she was concerned, it was clear that the city council was “being informed” rather than being asked to vote on the system. 

The city hoped the pilot could prove skeptics like her wrong.

Upping the stakes

The formal launch of Smart Check started with a limited set of actual welfare applicants, whose paperwork the city would run through the algorithm and assign a risk score to determine whether the application should be flagged for investigation. At the same time, a human would review the same application. 

Smart Check’s performance would be monitored on two key criteria. First, could it consider applicants without bias? And second, was Smart Check actually smart? In other words, could the complex math that made up the algorithm actually detect welfare fraud better and more fairly than human caseworkers? 

It didn’t take long to become clear that the model fell short on both fronts. 

While it had been designed to reduce the number of welfare applicants flagged for investigation, it was flagging more. And it proved no better than a human caseworker at identifying those that actually warranted extra scrutiny. 

What’s more, despite the lengths the city had gone to in order to recalibrate the system, bias reemerged in the live pilot. But this time, instead of wrongly flagging non-Dutch people and men as in the initial tests, the model was now more likely to wrongly flag applicants with Dutch nationality and women. 

Lighthouse’s own analysis also revealed other forms of bias unmentioned in the city’s documentation, including a greater likelihood that welfare applicants with children would be wrongly flagged for investigation. (Amsterdam officials did not respond to a request for comment about this finding, nor other follow up questions about general critiques of the city’s welfare system.)

Advertisement

The city was stuck. Nearly 1,600 welfare applications had been run through the model during the pilot period. But the results meant that members of the team were uncomfortable continuing to test—especially when there could be genuine consequences. In short, de Koning says, the city could not “definitely” say that “this is not discriminating.” 

He, and others working on the project, did not believe this was necessarily a reason to scrap Smart Check. They wanted more time—say, “a period of 12 months,” according to de Koning—to continue testing and refining the model. 

They knew, however, that would be a hard sell. 

In late November 2023, Rutger Groot Wassink—the city official in charge of social affairs—took his seat in the Amsterdam council chamber. He glanced at the tablet in front of him and then addressed the room: “I have decided to stop the pilot.”

The announcement brought an end to the sweeping multiyear experiment. In another council meeting a few months later, he explained why the project was terminated: “I would have found it very difficult to justify, if we were to come up with a pilot … that showed the algorithm contained enormous bias,” he said. “There would have been parties who would have rightly criticized me about that.” 

Viewed in a certain light, the city had tested out an innovative approach to identifying fraud in a way designed to minimize risks, found that it had not lived up to its promise, and scrapped it before the consequences for real people had a chance to multiply. 

But for IJmker and some of her city council colleagues focused on social welfare, there was also the question of opportunity cost. She recalls speaking with a colleague about how else the city could’ve spent that money—like to “hire some more people to do personal contact with the different people that we’re trying to reach.” 

City council members were never told exactly how much the effort cost, but in response to questions from MIT Technology Review, Lighthouse, and Trouw on this topic, the city estimated that it had spent some €500,000, plus €35,000 for the contract with Deloitte—but cautioned that the total amount put into the project was only an estimate, given that Smart Check was developed in house by various existing teams and staff members. 

For her part, van der Vliet, the Participation Council member, was not surprised by the poor result. The possibility of a discriminatory computer system was “precisely one of the reasons” her group hadn’t wanted the pilot, she says. And as for the discrimination in the existing system? “Yes,” she says, bluntly. “But we have always said that [it was discriminatory].” 

She and other advocates wished that the city had focused more on what they saw as the real problems facing welfare recipients: increases in the cost of living that have not, typically, been followed by increases in benefits; the need to document every change that could potentially affect their benefits eligibility; and the distrust with which they feel they are treated by the municipality. 

Advertisement

Can this kind of algorithm ever be done right?

When we spoke to Bodaar in March, a year and a half after the end of the pilot, he was candid in his reflections. “Perhaps it was unfortunate to immediately use one of the most complicated systems,” he said, “and perhaps it is also simply the case that it is not yet … the time to use artificial intelligence for this goal.”

“Niente, zero, nada. We’re not going to do that anymore,” he said about using AI to evaluate welfare applicants. “But we’re still thinking about this: What exactly have we learned?”

That is a question that IJmker thinks about too. In city council meetings she has brought up Smart Check as an example of what not to do. While she was glad that city employees had been thoughtful in their “many protocols,” she worried that the process obscured some of the larger questions of “philosophical” and “political values” that the city had yet to weigh in on as a matter of policy. 

Questions such as “How do we actually look at profiling?” or “What do we think is justified?”—or even “What is bias?” 

These questions are, “where politics comes in, or ethics,” she says, “and that’s something you cannot put into a checkbox.”

But now that the pilot has stopped, she worries that her fellow city officials might be too eager to move on. “I think a lot of people were just like, ‘Okay, well, we did this. We're done, bye, end of story,’” she says. It feels like “a waste,” she adds, “because people worked on this for years.”

In abandoning the model, the city has returned to an analog process that its own analysis concluded was biased against women and Dutch nationals—a fact not lost on Berkers, the data scientist, who no longer works for the city. By shutting down the pilot, he says, the city sidestepped the uncomfortable truth—that many of the concerns de Zwart raised about the complex, layered biases within the Smart Check model also apply to the caseworker-led process.

“That’s the thing that I find a bit difficult about the decision,” Berkers says. “It’s a bit like no decision. It is a decision to go back to the analog process, which in itself has characteristics like bias.” 

Chen, the ethical-AI consultant, largely agrees. “Why do we hold AI systems to a higher standard than human agents?” he asks. When it comes to the caseworkers, he says, “there was no attempt to correct [the bias] systematically.” Amsterdam has promised to write a report on human biases in the welfare process, but the date has been pushed back several times.

“In reality, what ethics comes down to in practice is: nothing’s perfect,” he says. “There’s a high-level thing of Do not discriminate, which I think we can all agree on, but this example highlights some of the complexities of how you translate that [principle].” Ultimately, Chen believes that finding any solution will require trial and error, which by definition usually involves mistakes: “You have to pay that cost.”

Advertisement

But it may be time to more fundamentally reconsider how fairness should be defined—and by whom. Beyond the mathematical definitions, some researchers argue that the people most affected by the programs in question should have a greater say. “Such systems only work when people buy into them,” explains Elissa Redmiles, an assistant professor of computer science at Georgetown University who has studied algorithmic fairness. 

No matter what the process looks like, these are questions that every government will have to deal with—and urgently—in a future increasingly defined by AI. 

And, as de Zwart argues, if broader questions are not tackled, even well-intentioned officials deploying systems like Smart Check in cities like Amsterdam will be condemned to learn—or ignore—the same lessons over and over. 

“We are being seduced by technological solutions for the wrong problems,” he says. “Should we really want this? Why doesn’t the municipality build an algorithm that searches for people who do not apply for social assistance but are entitled to it?”

---

Eileen Guo is the senior reporter for features and investigations at MIT Technology Review. Gabriel Geiger is an investigative reporter at Lighthouse Reports. Justin-Casimir Braun is a data reporter at Lighthouse Reports.

Additional reporting by Jeroen van Raalte for Trouw, Melissa Heikkilä for MIT Technology Review, and Tahmeed Shafiq for Lighthouse Reports. Fact checked by Alice Milliken. 

You can read a detailed explanation of our technical methodology here. You can read Trouw's companion story, in Dutch, here.