SNOMED rf2 to json conversion

The docs cover the conversion of Snomed data version 20180131. For other verions there may be slight difference in the steps.

Assumptions

OpendJDK 1.8.0_131 installed. JAVA_HOME environ variable is set. mvn version 3 is installed. cloned repo of https://github.com/IHTSDO/rf2-to-json-conversion. SNOMED SNAPSHOT data downloaded. MongoDB version 2.6.9

Steps

Create the file config.xml as given below. And set the values of following elements, effectiveTime, expirationTime, outputFolder, folder suitably.

    <?xml version="1.0" encoding="UTF-8"?>
    <config>
        <processInMemory>false</processInMemory>
        <defaultTermLangCode>en</defaultTermLangCode>
        <defaultTermDescriptionType>900000000000003001</defaultTermDescriptionType>
        <defaultTermLanguageRefset>900000000000509007</defaultTermLanguageRefset>
        <normalizeTextIndex>true</normalizeTextIndex>
        <createCompleteConceptsFile>false</createCompleteConceptsFile>
        <editionName>International Edition</editionName>
        <databaseName>en-edition</databaseName>
        <effectiveTime>20180131</effectiveTime>
        <expirationTime>20190131</expirationTime>
        <outputFolder>Location of output folder</outputFolder>
        <foldersBaselineLoad>
            <folder>Location of SNOMED SNAPSHOT data.</folder>
        </foldersBaselineLoad>
        <modulesToIgnoreBaselineLoad>
        </modulesToIgnoreBaselineLoad>
        <foldersExtensionLoad>
        </foldersExtensionLoad>
        <modulesToIgnoreExtensionLoad>
        </modulesToIgnoreExtensionLoad>
    </config>

Compiling the code to create a conversion jar

If you have the precreated jar, you may skip this step. The jars can be found at https://github.com/IHTSDO/rf2-to-json-conversion/releases. For conversion of SNOMED version 20180131, rf2-to-json-conversion-1.3-jar-with-dependencies.jar was used.

Go to folder rf2-to-json-conversion and run $ mvn install

The above command will create a jar called rf2-to-json-conversion-1.2-jar-with-dependencies.jar.

Note: At the time the above file was created. Later, the created file may have different name.

Conversion process

$ java -Xmx8g -jar rf2-to-json-conversion-<version>-SNAPSHOT-jar-with-dependencies.jar config.xml The command will create the converted file in the output folder specified in config.xml.

Note: The process may take long time, depending upon the RAM and CPU available.

Import Process

Go to folder rf2-to-json-conversion/scripts. Run $ ./import.sh <mongodb host> <database_name> <collection_name> e.g. ./import.sh localhost en-edition 20160131

The script assumes the files generated by conversion process resides in the same folder as import.sh.