The Quality Control portion of a document review is a vital step in the review process. It provides an opportunity for your project managers and yourself as a team leader to ensure your reviewers are well-versed in your subject matter and tagging the relevant documents correctly. It also allows you to check no privileged documents have slipped through the cracks and make sure the quality of your review is high.
In most cases the QC procedure is active throughout the entirety of the review, making it easier to spot and address inaccuracies as quickly as possible so as to protect the efficiency and integrity of your review.
As an eDiscovery provider that has been active in the industry for nearly two decades, we have seen our fair share of reviews and have built a solid QC workflow we can apply to most cases we see. Of course, each case has its own intricacies and our workflow is easily adapted to fit the specifics of each project we work on. Nonetheless, here are our best practices and what you can expect from Altlaw when it comes to QCing a review.
What steps are included in a review?
The first step in your doc review journey is Early Case Analysis. This is where we use many different tools to cull your documents to the (hopefully) few deemed to be most likely to be relevant. The tools we use include, but are not limited to, deduplication, keyword searching, email threading and concept clustering, but even after applying all of these, there may still be a need to review.
When you move on to the full review process it is common for your review to be split into first and second-level reviews. This helps keep the workload manageable and separates out what the reviewers are looking for in each review. QC reviews are then conducted alongside these to ensure the work product is accurate and consistent with the instructions set out in the Document Review Protocol.
Our QC process:
The first-level review is typically the most important review to QC as the paralegals, or outside reviewers if you're conducting a managed review, are just getting to grips with the case matter and the coding parameters. Ensuring your reviewers are aware of the coding fields and what documents fall into each category is the first step of a QC review and this is what your project managers will be looking out for in the first couple of days of your review. The standard review coding fields are:
- 1LR_Relevance (Relevant, Not Relevant, Unsure)
- 1LR_Privilege (Wholly Privilege, Part Privilege, Not Privilege)
- 1LR_Technical Issue (Yes/No)
- 1LR_Issues
- 1LR_Comments
We also track which reviewers have made which decisions and the dates these decisions occurred. This allows us to offer specific guidance tailored to your reviewers and their level of understanding.
We run QC reviews every day at the completion of the day's review, overseen by our review manager who then assists in the organisation of the documents that have been coded that day. A percentage of these documents will be included in the second-level review to reduce the likelihood of miscoded documents. There is also a chance that new evidence will come to light throughout the review process that makes previously discarded documents more relevant. The percentage of documents promoted to the second-level review is up to the client, but we would advise they take into account these factors...
- If your reviewers are fairly inexperienced or unfamiliar with the case matter we would recommend a higher percentage of documents to be promoted.
- If you would like a high level of accuracy in your review we would recommend a higher percentage of documents to be prompted.
- If you have a lot of time available for the second review we would recommend being more thorough and promoting a greater percentage of documents.
Typically your review manager will take your chosen percentage of documents and select the corresponding number of documents tagged as Relevant, Not Relevant, Wholly Privileged/Pert-Privileged, Not Privileged and similarly for each of the issue tags.
Our project managers will then look into those documents marked with the Technical Issue tag and see what can be done to resolve the issues. These are typically document visibility issues and are quickly resolved and reintroduced to the first-level review.
All documents marked as Relevant and Part-Privileged are then obtained and imaged in preparation for disclosure if required. At this stage, we would seek advice from the client on how they want images to be produced, eg. black and white, TIFFs or JPeg images.
All documents marked as Unsure are automatically sent to our QC reviewers so that the senior reviewers can take a look and advise the first-level reviewers on what to do going forwards if they meet similar documents.
Once this process is complete we then move on to the second-level review. The reviewers are supplied with slightly different coding fields and are able to see the previous coding decisions made in level 1. This allows the reviewers to agree or contest a coding decision. Similarly, with the first-level review the QC reviews are completed at the end of every day, however, no documents are promoted to the third-level review unless this has been previously agreed upon. As the reviewers for a second-level review are often much more experienced and involved in the case, it is deemed that these final coding decisions will be suitably accurate for culling or production.
Any further questions?
We understand that this is a reasonably complex and technical aspect of eDiscovery so please do not hesitate to reach out to us via our 'book a consultation' page to speak to one of our Altlaw experts about what steps we would recommend for your review.
