Smile, You’re on Facial Recognition

Untitled Document

Smile, You’re on Facial Recognition

Developing Technology Could Solve Patient Identification Issues

By Valerie McCleary, MS, RHIA, CCS

Many technological aspects of science fiction are seamlessly becoming a part of people’s everyday lives. The writings of Isaac Asimov, George Orwell, and Aldous Huxley shared what the authors envisioned as the future. Their stories transported readers to worlds that were limited only by the author’s imagination. Television viewers can also relate with shows like Star Trek, while movies like Back to the Future also envisioned possible futuristic and technological advances.

Many of those concepts have become reality. Consider space travel, drones, cell phones, biometrics, wearable devices, and the use of lasers. Automatic doors, video calls, and robots that vacuum carpets are daily conveniences of everyday life. Automakers are on the cusp of bringing self-driving cars to the mainstream. Alexa, Echo, and Google Home have brought artificial intelligence into our dwellings, listening in on personal family lives and responding on cue to inquiries and demands. While Alexa can’t make dinner, she can order meal delivery on command.

Images are captured everywhere today. Facial recognition computer software now exceeds a person’s ability to recognize faces in a sea of images. Facial recognition—no longer in the realm of science fiction—is a form of biometric technology that must be recognized and managed to its best advantage in health information management (HIM), and used in accordance with HIPAA regulations. Facial images are increasingly included in electronic health records (EHRs) as a means of identifying the correct patient receiving healthcare services, of guarding protected health information from identity theft, and as a diagnostic tool.

Facial recognition is not 100 percent accurate, but it is improving rapidly. Biometrics is predicted to quickly replace two-factor authentication and will speed up registration, information access, information management, and treatment processes. Facial recognition could potentially serve as a federated patient identity—more commonly referred to as a national patient identifier.

Many people agreed with author Jeffrey Rosen when he wrote, “Before Sept. 11, the idea that Americans would voluntarily agree to live their lives under the gaze of a network of biometric surveillance cameras, peering at them in government buildings, shopping malls, subways and stadiums, would have seemed unthinkable, a dystopian fantasy of a society that had surrendered privacy and anonymity.”1 Times sure have changed. Cameras everywhere are accumulating billions of images used in facial recognition systems. People are now accustomed and desensitized to the increasingly pervasive use of surveillance cameras. Sometimes cameras are easily observed, while others are concealed. Whether we like it or not, cameras are nearly everywhere people travel in their daily routines, and are making their way into healthcare.

Widening the Aperture: Cameras are Everywhere

Cameras are a helpful tool to thwart and solve crimes. Police body cameras are being introduced where budgets allow. Security cameras are recording 24/7 in many public and private industries, parks, libraries, banks, schools and colleges, shopping malls, churches, sports arenas, and city streets. Employee photo ID badges are worn in many industries.

Surveillance camera systems are now affordable for personal use. Homeowners are installing advanced motion-activated security systems with multiple cameras indoors and out, which are accessible remotely and online. Personal dash cameras are used as evidence in vehicle accident claims. Webcams are a standard feature on personal computers, tablets, and smartphones. Photos and impromptu “selfies” are shared on social media sites. The reader may be viewing this article on a device that has a camera. Billions of images serve as a vast data pool for facial recognition systems. Many EHRs in healthcare facilities, clinics, and physician offices also contain images for patient identification.

Point and Shoot: Facial Recognition and How It Works

Facial recognition is a form of biometric technology employing geometrical (or photometric) statistical methods to identify and compare a single facial image against other facial images in a database. An image of a person’s face is identified when a match is found. There are several different ways a facial match can occur when attempting to identify an individual using facial recognition. According to the authors of a recent white paper on the topic, “Facial recognition algorithms can be classified as appearance-based, feature-based, and texture-based.”12

“Appearance-based algorithms usually rely on the global semblance of features,” they explain. “Feature-based algorithms establish a relationship among facial features and perform matching. Texture-based algorithms rely on facial texture information.”

Facial recognition uses an individual’s facial image, which is considered a physical body trait of biometrics. People have become very familiar with other physical body traits such as fingerprints and DNA testing, thanks to many popular criminal investigative television shows. Other forms of physical body traits include iris recognition, hand vein pattern recognition, palm prints, and voice prints. Biometric identification continues to expand using physical body traits and evolving technologies. Industries are recognizing the advantages of biometrics and are including this item in their budgets.

Innumerable images can be scanned and reviewed much quicker than the human eye. This technology has greatly surpassed what can be accomplished by a human being. Facial recognition technology can now clear fuzzy pictures, pick out people in a crowd, and differentiate between genetically identical twins. Partial faces or profiles can also be reconstructed to produce a full-face view.

Image Stabilization: Moving to a Biometric Surveillance Database

Author Steven Nelson noted in an October 2016 article in U.S. News and World Report that “At least half of American adults are already in a police facial recognition network according to a report published by Georgetown University’s Center on Privacy and Technology.”3 Millions of images have been accumulated from driver’s licenses, the Federal Bureau of Investigation (FBI) database, and others. The United States is growing ever nearer to forming a single government biometric database for total surveillance. While the thought is shocking and objectionable to many, for law enforcement a single facial recognition database is a welcomed resource to apprehend criminals and protect the public. It is possible that the facial image for readers of this article already exists somewhere in a facial recognition database.

Exposure Time: Privacy Concerns

Health records are under attack due to the valuable demographic, financial, and medical history data they contain. Thankfully, privacy and security measures for health records have long been established. Covered entities that bill electronically are bound by HIPAA privacy and security rules. According to a Business Law Today article, “Biometrics are considered protected health information (PHI) under HIPAA. Biometrics are considered a patient identifier and must be removed from PHI when releasing de-identified information. Facial recognition opens a new area for temptation where an organization’s staff might use the system inappropriately.”4

According to the report published by Georgetown University’s Center on Privacy and Technology, “Currently state and local government agencies can conduct searches of their image databases without any safeguards in place.”5 People’s First or Fourth Amendment rights need to be considered in this situation. The article notes, “There is no state requirement for reasonable suspicion to warrant conducting facial searches. Periodic facial recognition system auditing is not required.”

While the Federal Trade Commission has issued best practice recommendations regarding use of facial recognition data, no formal laws or rules have yet been established.6 The Business Law Today article discussed how some states have started to develop legislation regarding biometric data and third party use.

A few states have started to develop legislation for third party use and collection of biometric data. These include:7

  • Collection and use of student biometric information (California and Illinois)
  • Collection by the government (Missouri, Maine, and New Hampshire)
  • Collection and use of biometric information by businesses (Illinois, Texas, Alaska, and Washington)

The public wants criminals caught, but many law-abiding American citizens’ images (civil government workers, driver’s license images) exist in those databases. Sensitive situations exist where privacy is the expectation. Consider the delicate circumstances of rape, child pornography or sexual trafficking victims, people participating in the witness protection program, confidential journalist sources, and domestic violence shelter residents. In some instances, protective limitations on facial recognition could save a life.

Lens Options and Lighting: The Challenges

As with any security system, there are those who will storm the fortress to try to gain access to the valued contents within. Biometrics have a high rate of accuracy, but they are not infallible. The report published by Georgetown University’s Center on Privacy and Technology notes that “Fingerprinting currently remains more accurate than facial recognition.”8Algorithms are improving, however. According to Hong Kong’s Polytechnic University, a three-dimensional contactless fingerprint identification system has been developed with an accuracy rate of approximately 97 percent and processes fingerprints in about two seconds.9 Contactless fingerprint systems avoid direct contact between the sensor and the skin’s surface, preventing fingerprint distortion when pressed against a hard surface. This technology would also be useful in healthcare since it would prevent the potential to transmit disease.

Algorithms do not factor in racial differences. How accurate are they in dealing with different races? A 2013 article featured on the PBS NovaNext website discussed how the facial recognition algorithms used to investigate the April 15, 2013, Boston Marathon bombing were unable to match the suspects’ faces to their names. Both brothers were recorded in the state’s driver’s license system, and one was in the FBI database. The brothers were identified by publishing their photographs in the media to elicit tips from the public. This low-tech method produced results where facial recognition had failed.

Facial images change as people age. In an article published by the American Council on Science and Health, author Erik Lief recommends new images be taken every four to five years.11 Consider the changes seen in a collection of school photos taken over the years. Driver’s license photos are retaken after several years. For another example, while MasterCard uses selfies to sign into accounts, only one faceprint is created. This biometric measurement could be lost as an identifier if that faceprint is ever compromised, or if an individual’s facial features change over time without updating the selfie.12

Images can change significantly due to changes in physical factors such as weight gain and loss of muscle tone. Cosmetic surgery is very common and affordable today, and also would affect facial recognition. Rhinoplasties, rhytidectomy (face lifts), chin and cheek implants, and brow lifts can significantly alter facial appearance. Removal or addition of facial landmarks such as moles and tattoos can also affect facial recognition.

These shortcomings must be noted carefully by healthcare organizations looking to use facial recognition, since misidentification of patients could cause great harm.

Focus the F-Stop: Facial Recognition Technology Applied to Healthcare

Though it has its limitations, facial recognition technology promises smoother and safer patient access and information sharing in the near future. The move to increased technology, interoperability, and national health information exchange will lead to improved individual and population health.

The United States has long sought a workable national patient identifier. The College of Healthcare Information Management Executives (CHIME) began a competition called the National Patient ID Challenge with an eye-popping $1,000,000 prize. In November 2017, however, the challenge was suspended since the expected results were not achieved. Most of the top contenders attempted a solution using biometric technologies.

Facial recognition can prevent medical identity theft, where a person steals another’s health insurance, demographic, and financial data for personal or financial gain. Identity theft can:

  • Involve blackmail, using sensitive data from the patient’s medical history (HIV status, past abortion, etc.).
  • Cause an overlay of a thief’s medical data with the victim’s medical data. Overlays can result in accidental treatment errors, especially if one of the parties has an allergy.
  • Exhaust a patient’s health insurance limits.
  • Ruin a patient’s credit.

Facial recognition is also uniquely helpful in the healthcare setting by enabling patient identification (and the corresponding medical history) for an unresponsive person, when the use of fingerprints is not possible (burn victim, amputation), or for a person with mental illness or memory issues.

A National Human Genome Research Institute study revealed that facial recognition enables physicians to diagnose genetic diseases such as Down syndrome and DiGeorge syndrome.13This new diagnostic tool may facilitate early medical interventions.

Press the Shutter Button: Facial Images Improving HIM

New facial recognition products are appearing on the healthcare scene. Imagine the ease with which a patient’s image can bring up their file in the EHR using facial recognition software. There will no longer be a need to distinguish between similar patient identities to select the correct file.

In addition to using the software to identify patients, this technology can also be applied to grant or deny medical staff access to a patient’s protected health information. Imagine never having a problem again when you lose a key card or forget a password. According to a blog post published by facial recognition software vendor RightPatient, “Biometrics has already proven itself as a more secure alternative for accurate identification in healthcare. In fact, 69% of 16 to 24-year-olds recently polled indicated they believe biometrics will be faster and easier than passwords and PINs and half of those foresee the death of passwords by 2020.”14

NextGate’s facial recognition software, for example, claims to simplify registration, flag fraudulent activity, and eliminate the creation of duplicate records.15 If facial recognition software can actually live up to the expectations, it is possible that duplicate records may be a thing of the past for healthcare providers. Reducing fraudulent activity will save money, as well as better protect patients and their healthcare data.

These exciting facial recognition advances result in greater efficiency, security, safety, and accuracy in collecting and managing patient data. These are welcome technological changes to the HIM profession as it progresses into the future.


  1. Rosen, Jeffrey. “A Watchful State.” The New York Times Magazine. October 7, 2001.
  2. Singh, Richa; Mayank Vasta; and Afzel Noore. “Effect of Plastic Surgery on Face Recognition: A Preliminary Study.” White paper.
  3. Nelson, Steven. “Half of U.S. Adults Are in Police Facial Recognition Networks.” U.S. News and World Report. October 18, 2016.
  4. Claypoole, Ted and Cameron Stoll. “Developing Laws Address Flourishing Commercial Use of Biometric Information.” Business Law Today. May 2016.
  5. Garvie, Clare; Alvaro Bedoya; and Jonathan Frankle. “The Perpetual Line-Up: Unregulated Police Face Recognition in America.” Georgetown Law’s Center on Privacy & Technology. October 18, 2016.
  6. Claypoole, Ted and Cameron Stoll. “Developing Laws Address Flourishing… .”
  7. Ibid.
  8. Garvie, Clare; Alvaro Bedoya; and Jonathan Frankle. “The Perpetual Line-Up... .”
  9. The Hong Kong Polytechnic University. “Accurate Contactless 3D Fingerprint Identification System Developed.” ScienceDaily. March 30, 2017.
  10. De Chant, Tim. “The Limits of Facial Recognition.” NovaNext. April 26, 2013.
  11. Lief, Erik. “When Do Facial Recognition Photos Stop Working.” American Council on Science and Health. March 2, 2017.
  12. Claypoole, Ted and Cameron Stoll. “Developing Laws Address Flourishing… .”
  13. Mjoseth, Jeannine. “Facial Recognition Software Helps Diagnose Rare Genetic Disease.” National Human Genome Research Institute. March 23, 2017.
  14. Trader, John. “In Your Face: The Future of the Federated Patient Identification.” RightPatient blog. October 7, 2015.
  15. NextGate. “Facial Recognition—Transforming the Patient Experience: One Face at a Time.”

Valerie McCleary ( is adjunct faculty at Southern New Hampshire University.