Not logged in - Login / Register

View History

Sensitive Data Discovery

Data definition and discovery process

When a person is tasked with data anonymization the first thing s/he does is try to understand what subset of data s/he needs to address with masking. Basically, the first question is "what is sensitive data"?

The sensitive data definition

The term "Sensitive Data" or "PII" (personally identifiable information" or "PHI" in heath, protected health information, stands for the data that describes a person in a specific way, with certain attributes. The knowledge of the values of these attributes allows other people to re-identify that specific person among other people.

For example, the knowledge of Social Security Number allows learning a lot of things about a person. Social Security Number invariably is used in multiple systems during this person's life and is unique. The SSN value in the wrong hands can lead to false credit card applications, fraud medical claims, and exposure of public information about students.


There is a black market for stolen PII. Each element has its own price -for the very reason that it helps to earn the money in illegal ways. Besides commercial vendors, FBI and other government law enforcement entities take issue very seriously. People committing fraud get harsh sentences


Even if not with fraudulent intentions, compromising one's privacy is not desired. It is quite possible that a person would not want their employer, neighbours and sometimes even family members to find out about their health issues. Recent stolen data about extramarital affairs from Ashley Madison's site exposed a lot of people, and no matter how questionable the ethics of these people or behavior was, it cost a lot of ruined careers and even suicides

PII domain

Let's consider how we define attributes in the domain of sensitive data in terms of person's privacy and de-identification.

The common sense dictates that the more an attribute contributes to the unique description on a person or a company, the more important it is. The common sense also dictates that among attribtues there will be some unique identifiers ( either biological or societal) and there also will be a combination of non-unique identifiers that will describe a person or a company uniquely.

This concept, well popularized by Dr. Khaled El Emam in his book and on the site, takes roots much deeper, namely, with Codd and Date's definition of domains and keys. This concept of uniqueness, well known for fifty or so years in computer science and mnemonized by the saying “[Every] non-key [attribute] must provide a fact about the key, the whole key, and nothing but the key" for normalization (3nf) being applied to the domain of the person - provides for the core of Personally Identifiable Information Model. All the concepts of k-anonymity, l-diversity and t-closeness have roots in the definition of the candidate key.

Unique identifiers

While SSN, Passport #, Driver's Licenses are guaranteed uniqueness identifiers in the societal domain, fingerprints, irises, and genetic codes are considered unique enough in the domain of the biological markers.

Non-unique identifiers

The non-unique identifiers are those that in combination will make a person unique - and among them, usually name, gender,date of birth and place of birth as well as current address with all of its elements provide statistically significant identification of a person. The rest of identifiers could also be used in attacks including phone numbers, urls, IP addresses, VINs, company names, ethnic origins, and other data. Latanya Sweeny pioneered the notion, and one can find out their own "uniqueness" on her lab's page

Of course if you know some data about the person, you could deduct other data. For people working in the same organization, for example, the person's position (title) will limit number of subjects to a smaller circle of identifiable targets. Thus adding for example gender, and even the first name to the title of engineer, might bring you just several people - so if one has access to some of HR data, one can very well identify a person within the organization.

Industry domains

The Health industry defined the minimum number of attributes for a domain in their "Safe Harbor" list of attributes. The process of finding data in the systems and files across enterprise is called "sensitive data discovery". However, Safe Harbor does not constitute ALL of the attributes in the system, just the most common ones and there is another method called "expert determination". The guidelines for the method are defined in the HHS document

Other industries and countries do not define their domains in such details. However, with regulations in place, it is up to the practitioners to work with such domains in their "expert determinations". The specific attributes of e-commerce would be credit card numbers, and in financial industry, PAN (primary account numbers), credit scores, etc.

Industry trend

There is a variety of the companies on the market that come up with the automated tools for PII data discovery that would do a job to a satisfactory degree. HushHush is coming with such a tool in a second quarter of 2016.