When the Nazis came to power in the 1930s, they were quick to adopt early computer hardware to process huge data sets collected from census and national registration programs. For them, mass murder became a big data problem that was solved with the help of the then-innovative technology of punched card sorters and tabulators. It’s a tragic history where data about race, ethnicity, religion, and political beliefs became a tool of genocide. But it also led to the creation of a new generation of laws that establish the basic human right of data privacy.

In May 2018, the European Union’s groundbreaking General Data Protection Regulation (GDPR) went into effect. Normally a wonky data privacy law would pass under the radar of the public. And one coming out of the E.U. Parliament, which represents 27 member countries, would seem even less likely to be noticed by non-Europeans. In fact, the opposite was true. 

The GDPR made news in the U.S., because it has deep implications for American companies that market to E.U. consumers. It has even influenced California’s own innovative privacy law, the CCPA. And we, as ordinary Internet users, have felt its impact whenever we browse and are forced to click on the ubiquitous banner asking us for permission to access Web cookies. 

As someone who considers himself fairly well informed on privacy and data security policies (having blogged on this subject for too many years to admit), I was surprised to learn about the modern European history that directly influenced the genesis of the GDPR. And this history holds an important lesson for us about the unrestricted use of technology to gather information from the public. 

While we may think that giving powerful entities like Facebook and Google access to personal data is a uniquely 21st-century occurrence, Nazi-era Germany arrived at a very similar crossroads prior to World War II. When the Nazis came to power in the early 1930s, they collected data related to ethnicity, religion, and medical conditions. This highly personal data was then processed by early 20th century computer technology and ultimately fed into the Nazi’s killing bureaucracy. The statistical data informed the German leaders in their planning for deportations and murder of the “unfit” – Jews, Gypsies, and other “racially inferior” groups, as well as anyone who didn’t fit the Aryan profile.

Nazi Data Scientists

In today’s anything-goes tech world, big data is a multi-billion dollar industry, and data scientists crow over their abilities to analyze enormous data sets for insights into the consumer mind. In Nazi Germany, there was a similar excitement about learning the statistical makeup of the German Volk. And one German in particular, Heinrich Himmler, was very enthusiastic about using data science for the “strengthening of German culture.”

Himmler became the head of the infamous Shutzstaffel or SS, the internal military force charged with protecting the Nazi state. The coldly bureaucratic Himmler was a fanatical admirer of Hitler, and his SS was responsible for organizing the concentration camp system and carrying out the “final solution.” He, along with others in the Nazi leadership, were focused on ridding the Volk of supposedly undesirable traits – low birth rates, independent thinking, avoidance of hard manual work, and other non-Aryan behaviors.  

Volkskartei registration card. (Credit: German Federal Archive)

In 1939, the Nazis took a major step forward in setting up their workflow process for Volk improvement with a comprehensive census of the population. In addition to questions about family size, employment, and ethnic and religious backgrounds, there was an infamous separate supplementary card. For the first time, the Nazi government obtained in the census non-anonymous information, including name and address. More significantly, the additional card asked questions about the religions of all four grandparents. 

The Nazi statisticians were obssessed with identifying “racial Jews” based on formulas from the notorious Nuremberg Law, which declared German Jews as non-citizens and imposed on them restrictions and punishments. Since a number of Jews had assimilated, the statistical census problem was to work out the percentage of “Jewish blood” even if the family had long ago converted. Then, officials would apply the Nuremberg criteria that defined “racially  Jewish” as having three or more Jewish grandparents. 

In short, the Nazis were faced with the data search problem of finding specific information on about 600,000 Jews, both practicing and assimilated, less than one percent in a population of 80 million.

“The only value of a human being, which is a direct object of statistics, is his economic value. In the money economy, this is the monetary worth of human labor productivity.” — Friedrich Zahn, Nazi statistician 

Nazi Census Leads to Mass Registration of Citizens

The census would not be the only source of data for the Third Reich’s bureaucratic machine. The Nazis also wanted to gather current individual information on German citizens as part of a system of police surveillance and military preparedness for a draft. Orders went out in 1938 to create a registry of individuals, or Volkskartei, with the goal of recording for each citizen a current address, date of birth, occupation, criminal convictions, and many other personal details.

Citizen registration was directed by the maniacal Reinhard Heydrich, head of the Nazi secret police force, better known as the Gestapo. Germans and others who fell under Nazi rule were supposed to carry registration cards at all times. The card holder’s name or other identifying information could then be linked to the complete records. In a typical World War II movie, when the stock Nazis officers ask for “your papers, please,” they are indirectly referencing Heydrich’s Volkskartei. 

Heydrich thought of these detailed records as a tool to implement “Germanification” in the newly conquered countries, and for the eventual selection and removal of inferior humans. He was brutal even by Nazi standards – Hitler said he had an “iron heart” – and he led a harsh campaign to crush resistance in Czechoslovakia. Known also as the Butcher of Prague, Heydrich was assassinated in 1942 when his motorcade was intercepted by resistance fighters.

Punched Cards: The First Digital Storage Device

Looking back at this from a perspective of almost 80 years, you have to be struck by the Nazi’s grandiose vision of possessing total information on the people under its rule, and their industrial-level dehumanization of anyone not fitting their crazed vision. 

Some of the descriptions I’ve read of their plans for mass data collection seemed disturbingly modern – a net with “no loopholes,” more than “an address book” – and not out of place in describing an Internet-era monitoring tool. There simply was no concept of privacy in the Nazi state, as humans were reduced to data points in a cost-productivity formula. Is it any wonder that decades later the Federal Republic of Germany, prior to the passage of the GDPR, had the strictest privacy law in all of the E.U.?

The other point that surprised (and shocked) me when looking at this dark history is that the Nazis, in fact, did have basic computer technology to sift through the enormous data generated by censuses and citizen registrations. Before USB drives, before floppy disks, and before bulky disk drives, there was the punched card, a paper-based storage medium (see below). It’s a  pre-digital technology from another era, but far more efficient than the manual processes it replaced.

   80-column punched card. (Credit: Ventriloquist, GNU Free Documentation License)

Punched cards and the machines to read and process them were invented by Herman Hollerith. In a now forgotten chapter of company history, his innovative technology was ultimately acquired by International Business Machines, or IBM, a company you may have heard about. IBM also had a disturbing role in this story. Its German subsidiary, known as Dehomag (for Deutsche Hollerith Maschinen Gesellschaft), was all too willing to sell this technology to the Nazi government. The Nazis used the punched cards – at the time also known as Hollerith cards – to store data from the 1939 census, and Dehomag’s tabulating machines quickly processed the cards to get overall counts of Jews and other “undesirables.”

Early Spreadsheet Technology in Nazi-era Germany

Unfortunately for humanity, the 60-column Hollerith card and Dehomag technology were up to the job of handling the enormous census data set. Thousands of data clerks working at the Reich Office of Statistics were trained to record census forms on the card using special punchers that placed a rectangle hole in one of the 0 to 9 positions (above).  Columns could also be organized into multi-column fields to store three or four digit numeric data.

You can imagine these cards as rows in an Excel spreadsheet. The Dehomag card sorters and tabulators could then be used to perform spreadsheet-like operations on the data. Suppose the Nazi statisticians were interested in the number of unemployed Gypsies, a group the Nazis also considered “inferior.” They would configure a Dehomag D11 machine to perform the equivalent of an “AND” query to detect holes punched in the two relevant columns. This involved manually adjusting the electrical contacts on the machine so they would scan and select the targeted columns: When a circuit was completed because both holes had been punched, it would be added to the tally and shown on special clock-like meters. 

Tragically, the Nazis had the equivalent of late 20th century database technology in the 1930s.

Early Hollerith tabulator with dials representing counts of different categories.

(Credit: Arthur Shuster, Creative Commons)

While IBM had an expedient, profit-driven relationship with Hitler and the Reich, the larger issue was – and remains – authoritarians gaining access to private data on a mass scale. For example, even special stationery index cards could become tools of genocide in the Nazis’ hands.  While the German census provided aggregate information, the bureaucracy could then search the Volkskartei information on index cards (stored in large library-like buildings) to retrieve  individual records.

They were helped in their searches by a now little-known analog technology called edge-notched cards. These were once upon a time a common stationery store item and were simply ordinary index cards with columns of holes positioned at the top (below). As with the Hollerith card, each column represented a different category. The cards would all be aligned in a cabinet, and data technicians could manually pass a long needle-like rod through, say, the “social misfit” hole, and then pull up all the individual records that were caught. By using two rods it was even possible to perform an “AND” operation. To the Nazis, quaint stationery cards became another means for totalitarian oppression.

Notched-edge card’s holes implement analog data selection. (Credit: Daniel MacKay, Public Domain)

Privacy As a Human Right in the European Union

The Nazis’ obsession with detailed personal data and its consequences informed a new generation of EU policy makers. In contrast to the U.S. Constitution, the European Union’s Charter of Fundamental Rights (which became law in 2009) gives its citizens a personal right of privacy as well as a right to protection of their data. The Charter explicitly states that personal data can not be used without consent. There’s no equivalent right of data privacy in the U.S. legal framework. 

Based on the Charter, the GDPR then intentionally carved out a special category of sensitive data related to ethnicity, political opinion, religion, health status, and sexual orientation. Its collection is generally prohibited. And even under circumstances where it is permitted, the GDPR has strict requirements for organizations to take to protect this critical data. 

Article 8 of the European Union Charter of Fundamental Rights states: (1) Everyone has the right to the protection of personal data; (2) Such data must be processed fairly for specified purposes and on the basis of the consent of the person concerned or some other legitimate basis laid down by law.

Even more empowering (and a major sticking point for U.S. companies) is the GDPR’s “right to be forgotten.” While there are some qualifications, E.U. citizens can demand that companies erase all data related to them. Back in the U.S., the California Consumer Privacy Act (CCPA) has also adopted this right not to be seen by corporate bureaucracies and data brokers. It is the first and so far only state in the U.S. to implement this GDPR-inspired idea as a law.

The more important point is this: Though technologies will always be evolving, we can prevent, through laws like the GDPR and CCPA, organizations from obtaining unrestricted access to personal data – and using it to make decisions that affect our health, employment, and well-being. Will there eventually be a US-styled GDPR law at the national level? It’s something to think about the next time you agree to have a site collect your Web cookies.