Last Friday saw the release of the “Valukas Report”, named after Anton Valukas, the report’s author and Chairman of Chicago law firm Jenner & Block.  The report – commissioned by a US bankruptcy court as part of the Lehman Brothers bankruptcy, the largest bankruptcy in US history – is an interesting read, as it sheds light on exactly why and how Lehman Brothers collapsed.  As the Financial Times (among others) noted, this particular story offered very few heroes alongside a bevy of goats; the report also addressed the involvement of blue-chip UK law firm Linklaters after no US law firm could be found that was willing to sign off on Lehman Brothers’ aggressive “Repo” accounting practices.  At 2,200 pages and a cost of $38 million to complete, the Valukas Report has to be one of the more expensive reports ever drafted.

From an eDiscovery perspective, another interesting element of the report was how Mr. Valukas and his team went about searching, reviewing, coding and analyzing ESI of all different types in their effort to get to the bottom of why Lehman failed.  As articulated in an excellent post by The Posse List, the primary tools used by the examiners were linear review products from CaseLogistix and Stratify.  The review team was comprised of some 70 contract attorneys supplemented (and undoubtedly supervised) by Jenner & Block’s own attorneys.  With linear review tools like Stratify and CaseLogistix, presumably the Jenner & Block attorneys directed a manual first pass review  (i.e. each and every document is reviewed by a contract attorney, often organized by custodian, timeframe and/or simple keyword search) before Jenner’s own lawyers subsequently conducted a more strategic review of the ostensibly “high value” documents uncovered during the manual first pass review. Statistics of this first pass review are as follows (courtesy of The Posse List):

• Lehman’s digital universe was comprised of an estimated three petabytes of data — roughly the equivalent of 350 billion pages
• 5,000,000 of these documents equaling 40,000,000 pages were collected, given to Jenner & Block and loaded into either CaseLogistix or Stratify
• 34,000,000 pages were reviewed (85% of all documents collected), of which 4.44 million documents (26,000,000 pages) were in Stratify and 340,000 (8,000,000 pages) were in CaseLogistix
• We do not yet know the amount of time spent on the review or its total cost

As the aforementioned post hypothesized, it would be interesting to know what the cost savings – not to mention time savings and improvement in review accuracy and quality – might have been had the Valukas review team utilized Explore-in-Place™ for ECA or Predictive Coding™ to automate the review process.  (We gave a Predictive Coding primer in a previous post; please go here to have a look).  For simplicity, let’s focus solely on the review stage, and within review the first pass review as that is where the bulk of the cost tends to be generated.  We also need a few assumptions to create an “apples-to-apples” comparison.  These assumptions come from our years of experience in the eDiscovery space and should be within the ballpark of most proceedings.

• Cost of a contract attorney: $75 per hour (typically between $50-$100/hour)
• Documents per hour reviewed: 70 (typically between 50-100/hour, depending on review type, document type and length)
• 70 contract attorneys would be able to review approximately 4,900 documents per hour
• Based on a 7 hour work day (8 hour day less 1 hour for break), this would mean 70 contract attorneys could review 34,300 documents per day at a cost of $42,000 per day
• Thus, 1 contract attorney reviewing 1 document would cost roughly $1.22
• (Note: this does NOT include a number of other potential costs, including supervising attorneys, litigation support, hosting, review platform, etc.; it simply measures an estimated cost for contract attorney review time for the first pass review)

Using the above assumptions, one can hypothesize that the Lehman review (5,000,000 documents) would have taken roughly 146 days (70 reviewers eyeballing 70 documents/hour X 7 hours/day = 146 review days).  Based on a typical calendar of 22 work days per month, this would mean the review would take approximately 7 months.  As for the cost, 70 attorneys working 146 days would cost over $6 million (70 attorneys @ $75/hour X 8 hours/day = $42,000 per day; 146 review days = $6,132,000).  The bottom line: a 7 month first pass review at a cost of over $6 million.

Next, let’s look at Predictive Coding.  As mentioned before, Predictive Coding automates the majority of the review process by using powerful concept search and auto-categorization technology to 1) find key documents quickly, automatically and irrespective of keyword used in a search or a document, 2) automatically prioritize all documents for review (from most relevant or important to least), and 3) provide a computer-generated “review” of most/virtually all documents in a collection.  The resultant benefits are twofold: ECA (Early Case Assessment) which is automated, keyword-agnostic and incredibly insightful even before review has started, and a far faster and less expensive review.

In most Predictive Coding-led reviews, only a small fraction of a collection is reviewed by attorneys; while the exact amount varies by law firm philosophy, client and case, a typical range might be 10-20% of documents.  (Not coincidentally, a rule of thumb in eDiscovery states that in most cases only 10-20% of documents are not duplicative or even relevant to the matter at hand; thus, Predictive Coding essentially finds and prioritizes this 10-20% of documents for attorneys to review while providing a computer-generated review of the remaining 80-90% of documents).  Taking an overly conservative approach, let’s assume contract attorneys will review 25% of Lehman’s 5,000,000 documents (or 1,250,000 documents) while Predictive Coding would handle the remaining 75% of documents (or 3,750,000 documents).  Holding all other elements the same, here are the metrics using Predictive Coding:

• Cost of a contract attorney: $75 per hour (same as above)
• Documents per hour reviewed: 70 (same as above)
• 70 contract attorneys would be able to review approximately 4,900 documents per hour (same as above)
• 1,250,000 documents would thus take 255 hours, which equals 36 days or 7 weeks or 1.66 months (using the same 22 work day per month calendar)

How do the two sets of results compare?

• Our estimate of the linear review with CaseLogistix and Stratify:
  • 85% of documents reviewed
  • 29 week review (7 months
  • Cost of $6,132,000
• Our estimate of a Predictive Coding-led review:
  • 100% of documents reviewed
  • 7 week review (1.66 months
  • Cost of $1,512,000

  In other words, even using extremely conservative estimates for Predictive Coding, this more sophisticated approach would have reduced the review timeline by 76% and would have saved $4,620,000 (for a cost reduction of 75%).  Additionally, every single document in the collection would have been reviewed – instead of only 85% – which tends to give clients an added level of comfort (e.g. that key documents weren’t missed). 

  We’ll leave it to you to decide whether or not these results are compelling.  We will, however, go out on a limb by saying that we’re pretty sure Lehman creditors would be interested in having almost $5 million more to play with.