Pennsylvania’s Facial Recognition System, with 25 Million more Faces than the Commonwealth has Residents


It’s been a while since I posted to the Techlaration, and I’m cranking it back up to document the protracted installation of a massive surveillance infrastructure in Philly between now and the Democratic National Convention next summer.

By Kenneth Lipp

For three years Pennsylvania State Police and other law enforcement agencies in the Commonwealth have been using a facial recognition software that generates 3-D images of suspects and searches them against a database, through the Pennsylvania Justice Network (JNET). Updated in September and again in December of 2012, the JNET Facial Recognition System (JFRS) began in 2006, with a database of about 3 1/2 million images from the Commonwealth Photo Imaging Network (CPIN) and other state and local law enforcement sources that was made available to more than 39,000 municipal, county, state and federal justice professionals.

A 2013 announcement of expansion highlighted an apparently very serious shortfall in available faces:

Pennsylvania has over 3.5 million criminal booking photographs for comparison in JFRS. Unfortunately, this represents only a fraction of the approximately 13 million residents in the state.

In May of 2013 JNET filled its face-gap with 35 million images from the Pennsylvania Department of Transportation’s (PennDOT) own facial recognition database and added its MorphoTrust engine to the JFRS search. The current image universe available to everyone with access to the state justice network – representing about 500 agencies – now has a population of 25 million greater than that of the Commonwealth.

The JFRS combines the three largest proprietary facial recognition engines: MorphoTrust, NEC’s NeoFace, and Cognitec’s FaceVAC. What that means is that three different algorithms or sets of algorithms are used to attempt to match a picture submitted with pictures in the database. There are only 3 basic algorithms in wide industry use, and a lot of the differences between software are in at which stage in the process a given algorithm is applied, and what kind of preparations are done with the raw data. It’s important to keep in mind that computers don’t see color and features, they just decode 1’s and 0’s in blocks and sequences.

The Animetrics ForensicaGPS platform operated by the JFRS attempts to solve one of the persistent problems of automated image recognition – subjects that won’t look straight at the camera. Facial recognition technology is powerful because computers are able to use a strictly defined metric, like the relative pixel distance between facial features or identifying marks like scars, to process a huge collection the same way every time.

But a partial profile and a straight-on shot of the same face look to a computer to have concrete differences that are in fact relative to the viewer. ForensicaGPS uses a preparation algorithm to generate a 3-D view from a 2-D picture to fill in gaps in perspective. Results are mixed, in my testing. You can try a basic version for yourself. Below is “Weedney Houston” before and after 3-D treatment.

IMG_9086 3421885956eb063257ebafad325651b1_filled

And here’s former Attorney General Eric Holder, from a picture taken at a conference in Philly in 2013. Take note that this is not identical to ForensicaGPS, which allows for much more fine-tuning than the online “ID-Ready” app.

100_3599 0bc5b77b8c3adc4d2b46aebde1a2e475_filled

I had less luck with this side-view of Mayor Michael Nutter

Screenshot - 07202015 - 08:52:59 AMa98e0dc0b4799f26272d290d2dac05d7_filled

JNET stresses to users that JFRS is for “investigative purposes” only and does not make positive identification, but that’s not really reassuring. There doesn’t seem to be any threshold for suspicion before one of the almost 40,000 people with access to the system can submit an image for search, allowing for easy fishing expeditions. It has the potential to be used, and to some extent is already used, like Automatic License Plate Reader (ALPR) technology, with the digital signatures from facial scans taken by passing camera vehicles checked against a “hotlist” of persons of interest, as scanned plate numbers are checked against hotlists of stolen vehicles, and scofflaws.

This is not conspiracy theory, anyone who has a Facebook account, for example, willingly submits her own photos and those of others for computational analysis and commodification.

In the ALPR industry it is all about who has the biggest database – read the embedded documents for language which treats collections of tagged image data like business assets. The term “enterprise solution” is s refrain throughout JFRS documentation. That’s especially the case in the first of the two documents embedded below, which is an html copy saved by Google of a Pennsylvania law enforcement working group Powerpoint presentation, that is no longer online.

JNET is the first law enforcement application of Animetrics ForensicaGPS in the US, where its niche has been the commercial market and retail applications.

The video below provides a sort of demonstration of the software, muting is recommended, the music is annoying, then creepy.


The Declaration has a Harris Hailstorm of public records requests in the works.

About Kenneth Lipp

Kenneth is a writer and researcher. He’s from Alabama, and will not apologize for it. He moved to Pennsylvania in 2012, but has been in love with Philadelphia since a late-night stroll down Ben Franklin Parkway to the Art Museum in July of 2011 with the love of his life. He is interested in telling Philadelphia’s dynamic and absolutely unique stories with the zeal of a constantly enamored newcomer. Kenneth is also passionate about government transparency and protection of whistleblowers, most notably PFC Chelsea Manning. His research and reporting on law enforcement and surveillance have been featured in various publications, including Rolling Stone (Meet the Private Companies Helping Cops Spy on Protesters) and Popular Science (Boston Tested Crowd-Watching Software That Catalogues People's Skin Color). His training is in both genetics and history and he likes the joke about being a helicase and unzipping your “genes.” He’s driven to know, and thinks you can handle, the truth. Follow him on Twitter @kennethlipp.

There are 2 comments

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s