Image Analysis - developing a critical eye for images

As image analysts, we often get presented with hundreds of images for a single case. A number that large, or even larger, can overwhelm the analyst. Where do you start with that many images?

When looking at a seemingly complex problem, I like to take a play from an old football coach of mine. He used to say, "Son, how do you eat an elephant?" As a young man, I initially thought of all manour of witty retorts; knife and fork, with a bit of curry, invite your mates over with a few pints, and whatnot. His answer was simpler than that; "One bite at a time," he would answer. Therein lies the answer to our problem as well. We can look at our big pile one image at a time, then look for commonalities, then look for differences, then look to see what can be automated or batched, then look to see what needs to be fine tuned, and so forth. 

We can create a systematic, step by step approach to any imaging problem and break down any complex job into a series of simple technical steps. Whatever problems we may encounter can be discovered in this way and an approach crafted to fix them, both technical and aesthetic.

So, take a moment to look at the images that need to be processed. Do they all suffer from the same problem? Start to plan your path for the images and assemble the necessary tools to get you from start to finish. One easy way to sort your images is by creating Stacks in Adobe Bridge. You can Stack images that suffer from the same problem as the first step in your analysis. Looking at your work flow in this way will help in developing your critical eye, your most valuable tool in analysing images.

In the next installment, we'll look closer at Adobe Bridge and the opening steps in the work flow.

What - How - Why

In a previous post, I made reference to my "What - How - Why" way of thinking in approaching the learning process. Let's take a look at that today.

What - The Apprentice

There are a great many people who have purchased image processing software over the years. There are many vendors out there selling image manipulation software from the extremely complex like Photoshop CS3 Extended to Photoshop Elements to Microsoft's Digital Image Suite to Paint. Each one has it's learning curve. Time and again, we find these well meaning folks saying things like, "... let's just fix it in Photoshop." I am not making a judgement here. I am just observing that these people know WHAT they want done, but they don't quite know how to accomplish it or why their chosen hardware / software combination works the way it does or is properly suited for the given task.

To put this into the forensic realm, a user at this level should not expect to be able to explain a process employed (the what) that he/she does not yet know how to do or why he/she chose to do it versus some other method.

From an employee management standpoint, this user should be assigned to work with someone at the Master level in order to increase their knowledge, confidence, and proficiency. A suitable testing methodology should be employed in passing this user on to the next level.

How - The Journeyman

There are those who purchased Photoshop CS3 Extended with a sense of what they want to accomplish, but lack any kind of training - formal or informal. Over time, they've managed to hit the correct keystrokes a few times, and Googled their way into a few tips and tricks. They may have even cruised through the help files or watched a video at Bridge Home. In short, they know what they want and now know HOW to accomplish it. To the extent that they wrote the steps down, they may even be able to repeat these steps later on.

In terms of forensics, this user can identify what he/she wants to do and at least one way to do it. Management may become comfortable with this user working a case by him/herself, but may still want to have a Master level employee check and sign-off on the work product. 

From a staffing standpoint, the Journeyman may be tasked with assisting in the instruction of the Apprentice. Additionally, the Master level employee may task this employee with additional study to add depth and breadth to the user's knowledge base. A suitable testing methodology should be employed in passing this user on to the next level.

Why - The Master

There is power in the why of things. Knowing WHY something works the way it does allows the Master level employee to not only repeat the steps, but also to explain these steps to others (forensics), to teach others, and to know when something is appropriate and when it is not. The Master level employee understands however, that his/her learning has not stopped. He/she must demonstrate leadership by committing to retain this status as Master by continually striving to improve, by continuing his/her education, and by keeping the channels of communication open so that there is an exposure to new ideas and approaches in the working environment.

In terms of forensics, this user can operate independently and/or lead a team of users of all levels. This user becomes the face of the unit in court and in the greater community as casework is presented and explained.

From a staffing standpoint, the Master level employee may serve as the first line of supervision and/or may hold a senior or supervisor title such as Senior/Supervising Forensic Video Analyst or Forensic Imaging Specialist 3.

With all of this in mind, we can use this model to assess ourselves - to take a critical look at where we are professionally and look for areas for improvement. Given a set of instructions or a desired outcome from an investigator or an attorney, we can look at an image and determine what needs to be done. Knowing what needs to be done, most of us can think of at least one way to accomplish the how - or at least know how to Google the what. Learning the why gives us the power to explore alternatives, to build in efficiencies, and to better explain our processes to the court, to our peers, and to the media. Therefore, The why becomes the essence of Forensic Photoshop and the glue that binds together the curriculum.

What is Forensics

Forensics, or Forensic Science can be defined as applying a broad spectrum of tools and techniques to answer questions about a particular piece of evidence in the legal system. LEVA has a nice definition of Forensic Video Analysis. The Forensic Photoshop workflow can be used by more than just video analysts and is not exclusively a subset of Forensic Video Analysis. The workflow can be used by Latent Print Examiners, Questioned Document Examiners, Chemists, Criminalists, and so forth.

The word “Forensic” comes from the Latin word “forensis” meaning forum. During the reign of the Roman Empire, prosecuting a criminal charge meant presenting the case before a group of citizens. Both the accused and the accuser would give speeches based on their side of the story. Having the best argumentation and presentation skills would ultimately determine the outcome of the case. In other words, the person with the best forensic skills would win.

Let's fast forward from the Roman forum to 1987: John and Thomas Knoll begin the sub-routines for what would become Photoshop. After showing their product around under various names, they finally reached an agreement with Adobe. Adobe wisely made the early decision to mass-market Photoshop, rather than reserve it for a few imaging specialists. Photoshop really starts to get noticed in 1994 with the release of version 3 - which adds layers. It was at this point that I, as a designer in a mid-sized advertising agency, first came on board.

In the next installment, we'll look at what I call the "what-how-why" method of mastery and how it relates to the Forensic Photoshop workflow.

The workflow concept - introduction

I've seen many different approaches to curriculum design both as an instructor (peer) and as a student (end-user). The most popular approach has been to give a brief overview of the topic, touching on the basics and giving little in-depth information. Others, including myself, have picked a few topics of importance, and spent the majority of time examining them in great detail. Another approach has been the "what's new" way of looking at things. As an example, Adobe sent an able instructor to the Art Center when CS3 came out to give all of the instructors an overview of the new features in the Suite. Not a bad plan for the day when all in the room are up to speed and just need the highlights.

For Forensic Photoshop, I want to do things a bit different. Giving credit where it's due: I love the way George DeWolfe organizes his workshops - in terms of work flow. It just seems to work, simply and effectively, to deliver complex information in a common sense way. There are a lot of similarities between a Fine Art workflow and a Forensic workflow; the main one being mastery of the subject. There are also some distinct differences; mainly focused in what you can't do to an image and testify in a Daubert hearing. I love the enthusiasm of Rick Miller of Adobe (and Photoshop Instructor at the Art Center). When Rick talks about a technique in Photoshop, you feel as if you are in an old time Revival Meeting and get swept up in his infectious emotions. He really knows how to hold a room's attention. Chris and John Russ ... I have only briefly met Chris and know John from his books, but what a family duo. They have an amazing ability to be able to break down seemingly complex tasks into simple and repeatable steps. By way of example, Chris wrote a script to take a series of still frames, load them into layers, assign each layer an appropriate opacity level - as a solution for frame averaging ... whilst sitting in a class on Photoshop. Examples like these motivate me. I want to get better each day. I want to achieve mastery of my field. I want to share these stories, these ideas, these techniques, this workflow with you.

I am a firm believer in the what-how-why model of instruction. [I know what I want to do (apprentice), I know how to do it effectively and efficiently (journey level), and I know why it works and why it's the appropriate way to do it (mastery).] I also like the see one - do one - teach one model from medical schools.

For Forensic Photoshop, I will combine all three into a concise and effective curriculum. You will get the workflow, you will get the what-how-why of each step in the workflow, and you will see one - do one - and teach one. This solid foundation will help you in your path to Photoshop mastery as well as being able to communicate the process during testimony.

In future posts, I will outline all of these steps and lay the foundation for the upcoming LEVA class as well as the soon to be published book.

LEVA Conference

I found out today that there are 15 people signed up for the LEVA Pre-Conference PHOTOSHOP: BASIC TO ADVANCED IMAGING TECHNIQUES class that I will be facilitating in Calgary, AB next month. This is great news. I am sure that we can find room for a few more folks.

I was talking with Chris Russ of Reindeer Graphics / Ocean Systems recently and e-mailing with George DeWolfe . With their advice in mind, I've reworked much of the lesson plans from my previous classes at NATIA and the Art Center College of Design into a streamlined workflow that I think will go smoother in the 16 hour setting at the LEVA conference.

In the next few posts, I want to get some of the material from the class posted so that the attendees can come prepared with a sense of what is going to be happening. We'll start with an outline and go from there. I'll also add the links to the downloads that will be needed; Photoshop mainly (for those who don't have CS3), and whatnot.

Blending Mode Math - Part 2

Screen Blending Mode (Alt+Shft+S)

Screen, part of the Lighten group, can be used to quickly lighten a dark image. Often times, there is a tremendous amount of detail that is obscured by shadow. A Screen self blend can be used to brighten the image or a selected area of the image.

Screen: X = 1- ((255-U)*(255-L))/255

Screen mode inverts the values of each of the visible pixels in the two layers of the image. (That is, it subtracts each of them from 255.) Then it multiplies them together, inverts this value again and divides by 255. The resulting image is usually brighter and sometimes “washed out” in appearance. The exceptions to this are a black layer, which does not change the other layer, and a white layer, which results in a white image. The mode is commutative; the order of the two layers doesn't matter.

Soft Light Blending Mode (Alt+Shft+F)

A Soft Light self blend dodges the light areas and burns the dark areas of an image. This can be helpful when used on an image with a “back lit” suspect. It has been one of my personal favorites. When trying to accomplish non-destructively what Shadows and Highlights can do, Soft Light is my tool of choice. Shadow detail can be found by using an inverted, desaturated, and blurred Soft Light self-blend.

Soft Light: X = (((255-L)*U*L)+ (L*R_s))/255

The Soft Light equation is complicated. It needs R_s, the result of the Screen mode equation (shown above). This one treats the Upper image as if it were the light source (a Soft Light) for the Lower image. So if the Upper Image, or blend colour, is brighter than 50% (has a numerical value greater than 127.5, which is half of 255), then the Lower image, or base colour, gets lightened subtly. If the Upper image's colour is darker than 50% (has a numerical value less than 127.5) then the Lower image's colour gets darkened a little. As stated above, it dodges the light areas and burns the dark areas.

Difference Blending Mode (Alt+Shft+E)

Difference is great for aligning images or stitching images together. It is also helpful in quickly comparing images of a scene to see what is missing or has been added. If there is no difference between the layers, the result is black.

Difference: X =|L-M|

Difference mode subtracts the pixel value of the upper layer from that of the lower layer and then takes the absolute value of the result. The mode is commutative; the order of the two layers doesn't matter. Because the order of the layers doesn’t matter, Difference examines the numerical values for colour in both the base and blend images, and subtracts the lower number (darker colour) from the higher number (brighter colour). The result colour has a numerical value which is the difference between the two values.

Well, there you have it. A quick but detailed look at the most common blending modes used in forensic video analysis. I certainly hope that this bit of information spurs thought and discussion on the issue of the science of forensic video analysis and the use of Adobe’s Photoshop within the analyst’s work flow. I have found Photoshop to be one of the most potent tools in my toolbox. I love finding new ways to employ my old and trusted friend. If there is some process that you use, take a second, put it down on paper, and send it in. Have a question? Ask. 

I try to attend any Photoshop class, whatever the level, because there is always something more to learn … or a different way of looking at the same problem. I always encourage Photoshop users of all levels to do the same. Look for opportunities to further your knowledge. Read the new Photoshop books as they come out. See if and how the information conveyed in the book can apply to your workflow. If you find something new and great, don’t be afraid to share. For my part, I will be teaching a 2 day Forensic Photoshop class at the 2007 LEVA Conference. I’d love to see you there.

References:

Beardsworth, J. (2005). Photoshop Blending Modes Cookbook. O'Reilly.

French, N. (2006). Adobe Photoshop Unmasked: The Art and Science of Selections, Layers, and Paths. Adobe Press.

GIMP Documentation Team. (2007). GNU Image Manipulation Program User Manual. Free Software Foundation.

Harrington, R. (2006). Understanding: Adobe® Photoshop®: Digital Imaging Concepts and Techniques. Peachpit Press.

Lawler, B. (2005). Official Adobe Print Publishing Guide, Second Edition: The Essential Resource for Design, Production, and Prepress. Adobe Press.

Long, B. (2007). Adobe Photoshop CS3 Beta First Look with Adobe Bridge and Camera Raw. Peachpit Press.

Pankala, B. (2006). Layer blending modes explained. Retrieved from TeamPhotoshop: teamphotoshop.com.

Reding, E. (2006). Adobe® Photoshop® CS2 Revealed. Course Technology.

Blending Mode Math

Colour in an RGB image is made of three colour channels (Red, Green, and Blue), each of which has a numerical value from 0-255 for any pixel in that image. RGB is an additive colour system. The colour black would have a value of zero, while white has a value of 255 for each pixel in each channel. You can see these numerical values displayed in the Info Palette (F8). 

The calculations for each of the RGB channels will be performed the same, so an examiner can look at one channel or all channels individually to verify his/her work.

Let’s begin with an example of how we’ll work with the math. Rather than use the more conventional terms of base layer and blend layer, I will instead try to make it a bit easire and use the terms upper and lower as a way to describe what we will be doing. For those who run away scared at mathematical formulae (or even the mere mention of math), I will follow each with an English explanation.

Example: X=U + L

For each pixel in the upper (U) and lower (L) layer, add each of the corresponding colour components together to form the X resulting pixel's colour. As described above, a pixel's colour value must always be between 0 and 255. 

Important tip: if X is equal to a negative value, then 0 is the result (Black). If X is equal to more than 255, then 255 is the result (White).

Normal Blending Mode

Normal is Photoshop’s default blending mode and has little to offer the forensic video analyst.

Normal: X=U

Simply put, the resulting pixel value is equal to the value of the upper layer’s pixel. Another way of looking at it is to say that the layer on top covers the layer or layers below.

Multiply Blending Mode (Alt+Shft+M)

Multiply can be used to quickly darken a bright image. How many times have we come across a video from a camera that was facing the sun or some other light source, opening the camera’s iris and washing out the resulting images? The images can be darkened quickly (Multiply is part of the Darken group) using a Multiply self blend. This effect can be performed on the whole image, or selectively using a layer mask.

Multiply: X = 1/255 (U*L)

Multiply mode multiplies the pixel values of the upper layer with those of the layer below it and then divides the result by 255. The result is usually a darker image. If either layer is white, the resulting image is the same as the other layer (1 * L = L). If either layer is black, the resulting image is completely black (0 * L = 0). Note: the mode is commutative; the order of the two layers doesn't matter.

In the next installment, we'll look at the Screen Blending Mode. Future postings will include a look at Soft Light and Difference Blending Modes as well as providing a list of references for further details on this complex subject.

Forensic Uses of Adobe's Photoshop

Forensic Video / Image Analysts, Questioned Document Examiners, Latent Print Examiners ... there is something in Photoshop for everyone. Creative Suite got things moving. Now, with CS3, we are doing more than ever with this robust suite of programs.

Clarify video stills, convert formats, remove unwanted colours or artifacts to reveal hidden info, compare and authenticate ... all this for the price of one program. Quite the bargain.

Working in the field of Forensics, we are tasked with being able to prove our work and with providing a repeatable result. In Forensic Video Analysis, the examiner can use a multitude of tools to capture and analyze video retrieved from crime scenes. There are many vendors competing in the video capture market. From Avid to StarWitness and everywhere in between, somewhere in the examination workflow there is a common thread among examiners. More often than not, an examiner will use Adobe'€™s Photoshop.

When working non-destructively within Photoshop, the examiner will employ layers. In looking back at how far we’ve come in such a short time, remember that layers were introduced when version 3 shipped in 1994. Upon opening the image, the examiner will duplicate the background layer and work with the resulting copy, thus protecting the original. From this point, any number of adjustments can be made to clarify the image.

 

Introduced along with the layers functionality in 1994 was layer blending modes. Within Photoshop there are a variety of blending modes used to combine the color of the pixels in the upper layer with those in layer or layers beneath it. In forensics, we are blending with a copy of the background layer, also known as a self blend. A layer's blending mode determines how its pixels blend with the pixels of any underlying layers. Some blending modes have tremendous value in making time saving corrections to color and lighting within an image. The results are there in an instant and your eyes are the judge of when the image looks right. But what is happening underneath? Do we know how the blending modes work and how to check their accuracy? While countless books and articles have been written about Photoshop’s blending modes, very few have focused on their underlying mathematics. We verify, prove and document so much about our workflow; how do we prove our results when we use blending modes? In the next installment, we'll take a look at the math behind the blending modes.