"Stuff I'm working on ..."

Latest note

by kaeru published 2021/11/21 17:42:55 GMT+8, last modified 2021-11-21T17:42:55+08:00

Using Google Photos Facial Recognition Feature for Investigations

by kaeru published 2022/10/05 14:41:00 GMT+8, last modified 2022-10-06T23:51:23+08:00
Google Photos image recognition feature can be used to privately build a database of photos to match and search for faces
1 DJBT5V79At0dNqA1uiGq5w.png
Photo import/organization applications such as gThumb on Linux simplify process of adding comments and modifying dates
1 Ip17z9wtxK3Z9LAk3J-hwA.png
Imported “charity” golfing event photo with clear faces, notes and dates of forestry officials and businessmen in Google Photos

When there is enough photos for facial recognition (around 5 faces of the same person) Google Photos will start offering you some faces to start assigning names. These are private, and labeling is up to you, even if you may not know their names during initial stages of investigations. Go ahead and label them as Person of Interest A, Malaysian Official 1 or the real names if you know them. You can relabel them at any time.

1 84iUrzy6duobx0UfYlqthQ.png
Search bar now shows recognized faces. Click on the arrow to manage recognized faces.

As you add more photos or a person, from search option of a specific recognized face, there will be dialog from time to time, to help you train and improve matching results.

1 9FEygAWoatSiipMXM87aqw.png

1 7e9_wY9Gv68sR_aO8zFGsQ.png
Simple manual training dialog to improve facial recognition
1 p9PRt6dMEBMYhGVq8mCglw.png
Example of search for photos of Najib Razak and Irwan Serigar together in 2019

For individual journalists using tools like this is simple and free, and most image management software online or desktop, provide similar features for you to use.

Large media organizations with photography library, should look into applying similar approach for the media management library software, as it would provide powerful tool for their journalists for their reporting and investigations.

Government Reports as Sources of Photos

While public figures are easier to find on-line, image searches of photos of senior public officials who avoid the limelight will not return any results if any. Information and photos of many senior public officials, including technocrats, special officers are harder to find, especially after a regime change or when word of a possible corruption scandal starts to leak out.

0 mLFL0tzWuB3Us-Dt.jpeg
Limited results for Google Image search for Special Officer to Prime Minister Najib Razak, Wan Ahmad Shihab Wan Ismail

Colourful government annual reports are not only a good source of information for investigations, but also for photos for our facial recognition needs. We only need to extract about 5 or more photos of our PEP of interest. Use the screenshot feature, to quickly take photos from PDFs.

In the following example, there are enough photos from one government annual report to extract a small corpus of photos of a senior public official.

1 c-wjTbGTpoDQK3fRFAuOjA.png
Extracting Faces from MARA Annual Report 2012 for facial recognition training

If you have copies of Malaysian government agency annual reports, especially from 2012–2015 that are no longer available, please share it with Sinar Project’s Malaysian Government Document Archive project that aims to keep a searchable public digital archive of as many government reports of interest as possible.