BDD Program - Rapid Pilot Project Creation

Document Purpose

The purpose of this document is to provide the instructions to create the BDD Program "Rapid Pilot Project". This project consists of an Enron subset, and a handful of documents to showcase our Review Accelerators (Translation, Image Labeling, and Transcription) totaling ~40K documents. This allows us to have a project for demo purposes.

The next step following this project creation is to view the recording of the standard Reveal Demo and access the demo script

Technical Documentation

To access all technical documentation within the Reveal Discovery Platform, please go to our Online Help Portal. For all information regarding Review, please visit our Reveal 11 Online Help Portal.

High Level Process Overview

Below provides the high-level illustration of this entire process.

Access to Load Machine

All Load Machine access is administered through Perimeter 81. To access a Load Machine, you can follow the high-level steps below.

1. https://landing.revealdata.com
2. Use your Reveal Universal ID to login
3. Find/click on the icon for your Load Machine by searching for the MSA.
4. If you cannot find or access your Load Machine, please email support@revealdata.com and ask for access.

Stage Test Data

There are two steps to stage the test data.

1. Download the test data to the Load Machine.
2. Copy/Unzip the data to the root of the Z:\ drive on the Load Machine.

Download Test Data

Follow the steps below to download the Test Data.

1. On the Pilot’s Load Machine, use the following link in Chrome to download the Test Data,
   
   1. https://revealdata-public-test-data.s3.us-east-1.amazonaws.com/_RapidPilot.zip

Copy/Unzip the Data

1. After downloading the zip file, copy it to the Z:\ drive of the Load Machine.
2. Unzip this file to the root of the Z:\ drive so it looks like the below.
   1. 
   2. 

Project Creation

Reveal Review Manager is used to administer your instance of Reveal Review. In Review Manager, you can do any number of functions associated with the administration of Review such as create databases, manage users, load large amounts of data, set analytics and more. To see more information about this, please see Reveal Review Manager Overview.

Create Company

The Reveal Company may already exist. If it does exist, please skip this step, otherwise create the Company Reveal using the steps below. If it does not exist, or you want to create a new Company, follow the steps below.

A Company is the highest level of organization in our permission structure. Please create a Company named Reveal.

1. Open Review Manager on the Load Machine
2. Expand Instance Setup
3. Click Companies
4. Click the Add New tab
5. Enter Reveal (or preferred name) as the Company name, and click Insert
   1. 

Create Client

The Reveal Client may already exist. If it does exist, please skip this step, otherwise create the Client Reveal using the steps below. If it does not exist, or you want to create a new Client, follow the steps below.

A Client is only associated with one Company. A Client can have multiple projects associated with it and is a way to keep a logical structure to the projects.

1. Open Review Manager on the Load Machine
2. Expand Instance Setup
3. Click Clients
4. Click the Add New tab, enter Reveal (or preferred name) as the Client name, enter a Client Number (typically a number, recommended to use date created in YYYYMMDD format), associate the Client with a Company, and Click Insert
   1. 

Create Rapid Pilot (RP) Enron Project

After creating or using a previously created Company and Client, follow the steps below to create a new Rapid Pilot.

1. Open Review Manager on the Load Machine
2. Expand Instance Setup, select Projects
3. Click the New tab
4. Enter the Project Name (use ‘RP_’ as the prefix, the client's MSA_Enron as the project name, i.e. RP_81010001_Enron), enter a Project ID (typically a number, recommended to use date created in YYYYMMDD format), associate the Company/Client, choose the Reveal InControl Standard Template, and UTC for Time Zone.
5. Click Create Case
   1. 

User Setup

After creating the project, there are a few things that must be done prior to importing the data.

Create Review Account and Add to Rapid Pilot Project

Create a Review account for yourself and make your user a Company Admin as detailed below.

1. Open Review Manager on the Load Machine
2. Expand Project Setup
3. Click Users
4. Click Add
5. Click Create New, fill in the Create a new User ID form, and click Create after you've entered in the requisite information.
   1. 
      
      
6. Choose the Review Account, and click Add
   1. 
7. Change Group to Administrators
   1. 

Company Admin Rights

A user must have Company Admin rights to create users or a project within Review. To make a user a Company Admin follow the steps below.

1. Open Review Manager on the Load Machine
2. Expand Instance Setup, and select Companies
3. Ensure the Company created associated with Rapid Pilot is the one selected
4. Click the Users By Company tab
5. Click Add User, and choose your user
6. After adding the user select the checkbox Is Admin.
   1. 

Checking for Company Admin Rights in Review

If a user does not have access to the Load Machine, but wants to check for Company Admin rights, the user can check within Review.

1. Log into Review
   1. You can get the URL to Review to any environment in Review Manager by expanding Advanced Options, click System Settings, and in the Setting column type 'indexserverserviceclient'. When copying this URL result, exclude '/review_indexserver' from the result.
2. Click Admin
   1. 
3. Click the Menu button in the upper left-hand corner and look for Company Admin.
   1. 

Import Data

We will import data into the project. This requires importing the DAT file into the system and creating indexes. To do this, follow the steps below. 

Import Mappings

A new field mapping will also need to be created to load the data.

1. Open Review Manager on the Load Machine
2. Expand Project Setup
3. Click Import Mappings
4. Browse to the following DAT file
   1. Z:\_RapidPilot\PilotDataProcessingFolder\Exports\0001_EXPORT01\EXPORT01_loadfile.dat
5. Click Match All Fields.  
   1. 
6. Sort by the field Table Name, click in one of the fields that say -- Select -- and find/remap only the fields that show here as they did not map properly (there will not be that many). The fields that appear under the drop-down are the fields to find under table name. Remap each of these fields.  When this is complete, there will be no more fields remaining in the drop down. 
   1. 
7. After mapping the unmapped fields, save this mapping as the Rapid Pilot Mapping
   1. 

Documents Import

To import the DAT file, follow the steps below.

1. Open Review Manager on the Load Machine
2. Expand Import
3. Click Documents
4. Choose the Rapid Pilot project
5. Choose the Rapid Pilot Mapping
6. Import 1
   1. Import Data File: Z:\_RapidPilot\PilotDataProcessingFolder\Exports\0001_EXPORT01\EXPORT01_loadfile.dat
   2. Click Import Data
   3. This will load 40,656 files.
      1. 
      2. 
      3. 
      4. 
7. Import 2

1. 1. Repeat the same steps from Import 1
   2. Use Import Data File: Z:\_RapidPilot\PilotDataProcessingFolder\Exports\0002_EXPORT02\EXPORT02_loadfile.dat
   3. This will load 86 files.

Documents Overlay

When this data set was created, there were a few fields that were not within the DAT file that are essential for visual analytics and AI. To get these fields populated, we will need to do an overlay or Update Data for the project in Review Manager. To overlay these fields, follow the steps below.

1. Open Review Manager on the Load Machine
2. Expand Import
3. Click Documents
4. Choose the Rapid Pilot project
5. Choose Update Data
6. Choose CSV
7. Overlay
   1. Import Data File: Z:\_RapidPilot\PilotDataProcessingFolder\Exports\0003_BRS_OVERLAY\BRS_OVERLAY.CSV
      1. Please note that this is a CSV file and not a DAT file.
   2. Click Update Data
   3. Click Match All
   4. We are only overlaying 4 fields. You will need to find the following two fields and deselect them from the overlay, as Match All will return 6 fields.
      1. Find the field SENT_DATE and deselect it (remove the checkbox from Import for the row) as we should be overlaying the field SENT_DATETIME and not SENT_DATE
      2. Find the field SUBJECT and deselect it (remove the checkbox from Import for the row) as we should be overlaying the field SUBJECT_OTHER and not SUBJECT

1. 1. Click OK
   2. This will update all files (40,742)
      1. 

Create Indexes

After importing the data, you need to index the data. Please only do this after Store Pending = 0 as seen below. As part of this process, we grab the native and text files from the load file, and move them to S3 in AWS, or the “store”. Also note there is 1 file that is a Store Error. This was a virus that was removed by AV, so this is expected. This process will take some time, so this may be a good time to take a break for about an hour.

Please note the project in the screen shot is for illustration purposes only, and you should be choosing your project you created.

1. Open Review Manager on the Load Machine
2. Expand Create
3. Click Indexes
4. Choose the Rapid Pilot project
5. Select both indexes
6. Click Index/Re-Index
7. Click Extracted, OCR / Loaded, Document_Metadata, and click OK
   1. 
8. Open a new Review Manager.
9. Click Indexes.  Select the same project.  Click Index/Re-Index. Click Native/HTML, and click OK

Create Document Folders

After kicking off indexing, run the following process in the Create section of Review Manager.

1. Document Folders
   1. Choose the Rapid Pilot project
   2. Select both data sets
   3. Change Document Folder Field to RELATIVEPATHPARENT
   4. Click Run
      1. 

Monitor Progress

1. You can monitor the indexing progress in the Review Manager, or you can go to Menu -> Jobs -> Index, select View All Jobs, and you will see the indexing process of the import. Make sure the counts in your project match the counts in the screen shot below except for Total Errors.
   
   1. 
2. Make sure the data loaded properly into AI by going to Menu -> Jobs -> AI Document Sync. If this shows complete
   1. 
3. Everything will be complete when the Candy Bar displays counts on the Dashboard, and that the index Brainspace connector overlay completed as well.  
   1. 
   2. 

Run Final Processes

These final processes will showcase the following key features of Reveal during the demo:

• Key Documents
• Review Accelerators (i.e., Image Label, Translation, and Transcription)
• Enable AI
• Field Profile

Key Documents

For demo purposes, the following documents highlight key differentiators in Reveal. To get to these quickly, it is recommended to add these to a Work Folder.  

1. Login to your project via your browser in Review
2. Next to Work Folders on top left, Click the ellipsis
   1. 
3. Click Add Folder -> Name Folder "Key Docs" -> Click to Grid View
4. Click into Search Toolbar at top right, Click the ellipsis
5.  Click Add Condition -> Type Begin Number -> Click Begin Number metadata field
6. Copy and paste the following Begin Numbers into the pop out window: 

   1. DEMO-037138

      DEMO-040676

      DEMO-040720

      DEMO-040742

7. Click Add to Search -> Click Search -> Results should look like documents below:
   1. 
      
      
8. Click Update -> Action to Take -> Check Folders -> Click dropdown to Work Folders -> Check off Key Docs -> Click Submit -> Confirm
   1. 

Review Accelerators - Label

Run Image Label on the following images for demo purposes.

1. Login to your project via your browser in Review
2. Select the Documents Folder Documents -> Dropbox -> Group Share -> Images, and this should return 49 files
   1. Note: If the top level folder does not return 49 files, click the dropdown arrow and initiate the process outlined below on each sub-folder.  
3. Click the Label button on toolbar
4. Click SUBMIT
5. You can monitor the progress of this job in the Jobs section (click Menu and choose Jobs). When the job is complete there will be a green check mark.
   1. 

Review Accelerators - Translate

There are two sets of documents for translation purposes of Japanese and Korean. Below illustrates how to translate the Japanese files. The process should be repeated for the Korean files as well.

1. Login to your project via your browser in Review.
2. Select the Documents Folder Documents -> Dropbox -> Group Share -> Foreign Languages -> Japanese, and this should return 30 files
   1. Note: If the top level folder does not return 30 files, click the dropdown arrow and initiate the process outlined below on each sub-folder.  

1. Click the Translate button
2. Next to Destination Text Set click the + Create new button, enter the following items, and click Submit.
   1. 
3. Back on the Translate Documents settings, choose the following options.
   1. 
   2. Note: If the Destination Text Set that was just created does not show up in the list of text sets to choose from, logout/login, and the text set will be available. 
4. Repeat this process for the Korean files (Documents Folder Documents -> Dropbox -> Group Share -> Foreign Languages -> Korean), but translate from Korean to English. Please add the Korean files to the same Text Set of Translation, and do not create a new Text Set.
5. You can monitor the progress of this job in the Jobs section (click Menu and choose Jobs). When the job is complete there will be a green check mark.

Review Accelerators - Transcribe

There is 1 deposition video to transcribe.

1. Login to your project via your browser in Review.
2. Select the Documents Folder Documents -> Dropbox -> Group Share -> Videos, and this should return 1 file
3. Click the Transcribe, and click TRANSCRIBE (leaving all default settings)
   1. 
4. You can monitor the progress of this job in the Jobs section (click Menu and choose Jobs). When the job is complete there will be a green check mark.

Review Accelerators - Show Transcription

After the deposition video completes, turn on the transcription to show it during a demo.

1. Login to your project via your browser in Review.
2. Select the Documents Folder Documents -> Dropbox -> Group Share -> Videos, and this should return 1 file
3. You can monitor the progress of this job in the Jobs section (click Menu and choose Jobs). When the job is complete there will be a green check mark.
4. Launch this file in Review by clicking on the row.
5. Click the CC Show Transcript button , and this will show the text next to the video when it is played. This setting will stick for all future demos of this file.

Enable AI - Add AI Model

Apply an AI Model from the Model Library for demo purposes. 

1. Login to your project via your browser in Review
2. Click into Project Admin -> Tags
3. Click Add Tags and Choices 
4. Enter and select the following information for the tag:
   1. 
5. Return to your main Project screen -> Click Supervised Learning
6. Click the gear icon by the AI Tag created (Junk)
7. Scroll through the AI Library Models and add the "Out of Office" AI Model
8. Click Run Full Process -> Click Save (at bottom)
   1. 
9. Click the Review Icon on left panel, return to dashboard. 

Enable AI - AI Responsiveness Mutually Exclusive Tag

Update the existing Responsiveness tag to enable an AI model for the tag.

1. Login to your project via your browser in Review
2. Click into Project Admin -> Tags
3. Click edit (pencil icon) next to Responsiveness, and enable AI
4. Select "Edit Access" next to Prediction AI Access and select Original Administrators to have access to view AI Prediction Scores 
   1. 

Enable AI - AI Issues Multi-Select Tag

Create a new multi-select tag named Issues, and enable an AI model for each choice within the tag.

1. Click Add Tags and Choices 
2. Label Tag Choice "Issues"
3. Chose Multi-Select Tag 
4. Add the following issues separately: Chewco; Jedi; Raptor
5. Select Prediction Enabled for each tag
6. Select "Edit Access" next to Prediction AI Access and select Original Administrators to have access to view AI Prediction Scores
   1. 

Enable AI - Finalize Tag Profile

1. Move the issues over to the Standard First Level Review Tag Profile that contains solely those tags or other tags relevant to typical workflow 
   1. 
2. Click the Settings button, and make sure the Original Administrators have access to the Field Profile.
   1. 

Enable AI - Populate Scores for AI Tags 

To get scores generated in the AI Tags, conduct the searches below, and bulk tag the results.

1. Login to your project via your browser in Review
2. Click into Grid View
3. In the top search toolbar enter in search for keyword Chewco
   1. 
4. Click Update from Mass Action Toolbar
5. Click Action to Take ->  Select "Include Tags" -> Select Responsive -> Select Chewco -> Click Submit -> Click Confirm in pop up window
   
   
   1. 
6. The AI Tagging needs both positive and negative examples to populate AI Scores accurately. To tag the negative documents follow the steps below.
7. In the top search bar, search for the word Chewco -> Add families from the tree expander -> search for Chewco again -> Change the boolean operator from OR to AND -> change boolean operator IS to IS NOT -> Click Search
   1. 
8. With the populated results, use the Mass Actions Toolbar again. 
9. Click Action to Take ->  Select "Include Tags" -> Select Non-Responsive -> Click Submit -> Click Confirm in pop up window
   1. 
10. Repeat these same actions for the issues tags LJM, JEDI, and RAPTOR (e.g. search for LJM instead, mass tag all docs from search result with respective tag) 

Field Profile

By default, field profiles are not assigned to roles out of the box. This makes the Default Field Profile inaccessible for all users. To fix this, follow the below steps, and you can use the Default field profile. 

1. Login to your project via your browser in Review
2. Select Admin 
3. Click Fields -> select Default -> click Edit
   1. 
4. Select Original Administrators -> choose Fields -> click UPDATE
   1. 
   2. Recommended Fields:
      1. Begin Number
      2. Begin Number Attach
      3. Master Date time
      4. Custodian
      5. Extension
      6. App Name
      7. Email Subject
      8. From 
      9. To
      10. CC
      11. BCC
      12. Responsiveness
      13. Issues
      14. Foreign Languages
5. Switch to Grid, and choose Default if not selected

1. 1.