Welcome to Dr. Olli's Blog!

Inmate Call Monitoring: Waste of Time or Gold Mine of Information? Part 2

Ok, so I left off at the point where we had all of this data and didn't know what to do with it. Let's recap the types of data most facilities have about inmates:

• telephone call recordings
• inmate visitation logs and recordings
• information contained in inmate management systems
• financial transactions in commissary accounts

Some facilities may have more data, such as emails to and from inmates and any information that is generated by detectives or internal affairs units.

The bottom line is that this can be quite a lot of information. Worse, information is typically in different formats and, more often than not, in many different locations. Telephone recordings are in one database, inmate management records are in another and investigative information might be in a database, or it could be on bits and pieces of paper scattered about detectives' desks.

To the uninitiated or those used to simple transactional databases this scene may sound like a horror story, but in reality it's a dilemma that crime and intelligence analysts in law enforcement and corrections face every day.  With this in mind, it is  really no surprise that the typical analyst spends 70% or more of his or her time gathering data and getting it into a format where it's useable, leaving at best 30% of their time spent doing actual analysis.  In an ideal world, these figures should be reversed.

What analysts and investigators need to be more efficient is some way to centralize all of this data, make it compatible and make it available.  You want the data to be available in such a way that the analyst doesn't have to worry about where to get it, how to put it together, what format it's in, or whether they'll have to spend countless hours correcting sloppy data entry or errors (a HUGE pet peeve of mine; I might write a separate article about this at a later point), matching up records and all those other frustrating tasks that make you want to pull your hair out and wish you'd taken a different career path.

The good news is, it's possible. The key is data warehousing. Data warehousing is a relatively inexpensive way to get information into one place. Best of all, it is certainly a lot cheaper and easier to do than trying to get multiple computer systems and databases to work together.

So, what is data warehousing? Well, one way of thinking about it is to consider each separate database (inmate records, phone system, visitations) as a producer of parts, just like automotive parts.  In this scenario, the data warehouse is the assembly plant for these parts. The consumer of the final product is the analyst or investigator.

One rather obvious advantage to warehousing is that instead of winding up with a bunch of car parts stored in different warehouses in different places, you can actually create a finished product, one that you can drive or, in our case, data that's in a format ready to be analyzed. In other words, analysts or investigators don't have to worry about figuring out how to build a car and how to put the different parts together; instead, they just go to the warehouse and pick up the type of car they need for a particular job. After all, if everybody who drove a car also had to be an engineer and know how to build a car, there wouldn't be many people driving cars. The irony is that for many crime analysts, this is the case. Apart from analytical skills, many modern crime analysts also need to know about databases, mapping, address matching, data transformation, data cleaning and in many cases, some level of programming. In fact, crime and intelligence analysts are arguably some of the most highly trained and well rounded folks in the criminal justice system. A well designed and implemented data warehouse can for the most part eliminate the need for those extra skills and allow analysts to concentrate on their primary objective: analysis.

Another advantage of warehousing is that it adds value to the pieces and parts that feed it. Just as a finished car is more valuable than the sum of its parts, so is the data contained in a warehouse.

So, from a practical viewpoint, a data warehouse does a lot more than just collect data from different sources and put them into a single location. It adds value to the original data by performing a number of operations on it, just as auto workers (or, these days, increasingly robots) do with car parts. For example, a data warehouse automatically attempts to correct certain types of errors in the data whenever a record is uploaded from a database, such as misspelled names or inconsistent addresses. This makes it easier to connect records from different databases and is usually done using a set of rules or something called "fuzzy logic", where algorithms are applied to try and match data. As a data warehouse grows, these rules also grow. With each install of our Call IQ software, for example, the data from the various databases feeding the warehouse are also analyzed. Every time an inconsistency is found that can be fixed automatically, a way to fix it is added to the rule base. As a result, all customers benefit from each new install. Naturally, it is still very important to ensure consistent data entry instead of relying on a data warehouse to fix things for the very simple reason that not everything can be fixed.

Another way that a data warehouse adds value is by collecting meta data. "Meta data" is really just a fancy term for "data about data", or statistical information about the data contained in the warehouse. For example, when looking at a phone call an investigator might want to know whether any other inmates have called the same number, or which other numbers the inmate has called. Rather than having to run a separate query, all the investigator has to do is look at the meta data, which is complied automatically. Also, a data warehouse can automatically detect linkages between people, places and organizations.

A data warehouse also adds value by transforming data. As mentioned earlier, corrections data can be in many different formats. So for example, audio recordings can be transformed into text, where they can be further analyzed using natural language techniques such as topic detection. Another example is attaching geographic coordinates to addresses so that they can be mapped.

These are just some of the advantages of data warehousing. A lot of well publicized efforts (and some rather spectacular failures) in criminal justice have centered on making different computer systems and their databases compatible rather than deploying a data warehouse. That's all well and good if all you want to do is retrieve information from a single interface, but for analytical purposes you still wind up with data in varying formats and standards, and you don't get meta data that encompasses all data within the various systems. Another downside is that analytical queries can be very complex and require a lot of processing power. That in turn tends to slow down production systems. Earlier in my career, many were the times that I got "taken out to the woodshed" for causing the records management system to come to a screeching halt with my queries.

The bottom line is that for most analytical and investigative applications where the data spans a variety of different platforms and databases, a data warehouse is the cheapest, most efficient way to go.

Inmate Call Monitoring - Waste of Time or Gold Mine of Information? Part 1

In my last post, I mentioned the virtual gold mine of information that criminal justice agencies are sitting on, but often are not doing anything with this information.  This is especially the case in the corrections industry.  So, what exactly is this gold mine?

 

Law enforcement agencies collect information about people, places and events.  They try to piece all this data together and make sense of it.  For example, suspects (people) have addresses (places) and commit crimes (events).  Correctional facilities also collect data about people (inmates), places (their addresses) and events (disciplinary actions, fights, criminal records, etc.).  But, when it comes to data, the one big difference between corrections and other law enforcement agencies is that inmates generate a ton of data about themselves while they are incarcerated: telephone calls.  Law enforcement agencies can only gather this type of information if they have a warrant, whereas most correctional facilities routinely record, store and - to a small degree - monitor calls made by inmates.

 

Being able to collect all of this data is both a blessing and a curse.  It is a blessing because if any inmate uses a phone call to commit fraud, intimidate witnesses, run his drug operation from within jail, or order reprisals against rival gang members, for example, the evidence is right there in the recorded telephone conversation.  However, this data is also a curse because inmates place so many phone calls each day that it is not humanly possible to listen to them all.  For instance, one of our largest facilities records about 1.5 million inmate calls a year that average 15 minutes each, resulting in a total of about 375,000 hours worth of recordings.  If we were to manually transcribe these telephone calls into text, the resulting amount of data equates to the following:

 

• - 10 DVDs
• - 80 CDs
• - 103 filing cabinets filled with single spaced, double sided typed pages
• - 52,000 books

 

That's a lot of data.  Consequently, only a very small proportion of inmate phone calls are ever monitored for criminal activity.  This is a persistent problem that cuts across all jurisdictional lines, including the Federal Bureau of Prisons (BOP).  For example, active monitoring by prison personnel covers less than 4% of all calls made in the federal prison system and is categorized by a 1999 BOP report as "virtually useless."  In state and local correctional facilities, that proportion is likely much lower.  The report goes on to comment that the software technology to automatically monitor phone calls was still being developed and not accurate enough to be useful ("Criminal Calls: A Review of the Bureau of Prison' Management of Inmate Telephone Privileges"; see http://www.usdoj.gov/oig/special/9908/index.htm).  Clearly, there is much room for improvement.  Unfortunately, the cost of increasing the proportion of calls that are monitored by humans is simply prohibitive.  For example, to monitor even 10% of the calls in the facility mentioned above would require over eighteen full time personnel.

 

So, why is it so important to monitor inmate telephone calls?  After all, the vast majority of inmates do not commit crimes while they are in jail.  Most of their phone calls are to friends and family and discuss everyday things that are of no interest to correctional officers.  Yet some inmates, particularly gang members and drug dealers, often continue to ply their trade from within prison and many of them do so by telephone.  It is estimated that in 2003 nearly 12% of the federal inmate population was involved in gang activity.  For state and local prisons the proportions were 13.4% and 15.6%, respectively (National Major Gang Task Force (2003): "A Study of Gangs and Security Threat Groups in America's Adult Prisons and Jails", Indianapolis).  This not only poses a significant threat to correctional officers but also a continued threat to local communities.  Gang members are often incarcerated close to their hometown and thus continue to have an influence in their communities (National alliance of Gang Investigators Associations: "2005 National Gang Assessment").

 

Concerns are not limited to gang activity, however.  The FBI has maintained that it is likely that Al-Qaeda will or already has attempted to convert inmates to radical Islam and recruit them for their cause (Pistole, John S.: "Statement for the Record before the Subcommittee on Terrorism, Technology, and Homeland Security of the Senate Judiciary Committee", October 14, 2003).  Indeed, both convicted shoe bomber Richard Reid and Jose Padilla, who was arrested for attempting to build and detonate a dirty nuclear weapon in the United States, were former inmates who had been converted to radical Islam while incarcerated, the former in Britain and the latter in a Broward County, FL jail (Office of the Inspector General, 2004: "A Review of the Bureau of Prisons' Selection of Muslim Religious Service Providers").

 

Sifting through all calls and finding those that contain evidence of criminal activity or other reasons for concern, such as a threat to commit suicide, is like trying to find the proverbial needle in the haystack.  Because all inmate phone calls are recorded, it is easy to go back after the fact and verify that an inmate did indeed commit fraud during a phone call, for example.  But that is really no different from looking at surveillance footage from a convenience store after a robbery was committed.  In both cases, the fact that the events were recorded did nothing to prevent them from occurring in the first place.  Like a tip from an informant about an impending robbery at a particular convenience store, investigators can really only rely on information from outside agencies or informants that an inmate may be committing crimes using a telephone, or randomly monitor calls and hope that every once in a while they will find something interesting.

 

So how do we mine the gold?  Stay tuned for my next article will explore some of the ways in which this might be accomplished.

Hi, I'm Dr. Olli, President, TruNorth Data Systems.

One of the things that has always fascinated me in working with and within the criminal justice system is the fact that so many agencies, whether they be police, corrections, courts or others collect vast amounts of data but often do little or nothing with it. Throughout my career I have focused on this issue and have been very fortunate to have had the opportunity to work in practical law enforcement, academia and private industry.

As such I have worn many "hats", whether as a crime or intelligence analyst, as an adjunct professor or lecturer, a systems scientist, a developer, project manager or contractor. Each of those roles has taught me many valuable lessons, perhaps the most important one of which is this: all parties - criminal justice practitioners, academics and private industry - must work together to solve not just the problem of sifting through and making sense of massive amounts of data, but also the myriad of other tough problems that face the criminal justice system in general and the corrections industry in particular.

Today's world is becoming more complex with respect to technology, science, socioeconomics, changing demographics, crime and many other issues. Moreover, the rate of this change in complexity is accelerating. All of these issues in one way or another affect corrections. What this means is that it is increasingly becoming impossible for any one party to thoroughly understand all of the issues as well as possible solutions. All the more reason for cooperation.

When I first started working with DSI about three years ago I was immediately struck by the mix of people working there: practitioners, some academics and, of course, private industry folks. I really think this is a winning combination that allows all of us to be more innovative and think outside of the box. It is also a key factor in the success of DSI's products.

What I would like to do in my contributions to this blog is focus on those aspects of information technology that pertain to corrections and its unique data environment, and examine them from various angles. Hopefully I'll be able to do that in such a way that it will foster discussion and participation among readers and encourage them to think about how we might work together to create more innovative solutions.