You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 27 Next »

Note : Scrubber 3.X is being ported to Apache cTAKES, this is an interim BETA release.



Intended usages

We recommend starting with the default properties and prebuilt train/test models.
The train and test models are anonymized feature sets (not text) generated by scrubber runtime.
scrubber.properties : all supported config options and features in one place.
Apache UIMA, Apache cTAKES, and WEKA distribution jars are loaded dynamically.


Default configuration

  • Developers can use Scrubber 3.X in "default mode" with the same settings as the provided train and test model files. Input and output settings are managed in scrubber.properties (file paths, database settings, method implementations).

Customize NLP pipeline

  • Scrubber uses Apache UIMA and Apache cTAKES packages, which together provide the NLP pipeline for lexical parsing and medical concept annotation. Generated feature sets are exported to the SQL database or model file (CSV, ARFF). The UIMA and cTAKES services used by Scrubber are defined and configured using scrubber.properties.

Customize Classifier

  • Scrubber can use different classifier implementations without recompiling the software.
  • By default scrubber dynamically loads the popular WEKA C4.5 decision tree classifier with multi-class support.


Software Features


Annotate text automatically

  • Annotate word tokens and redact PHI from physician notes
  • cTAKES lexical parsing and medical dictionary annotation
  • WEKA multi-class decision tree classifier (plugin default)
  • Protege UI support for human expert curators (reads output) 

Models (Train and Test )

  • Prebuilt train and test models can be imported to Weka (default), Matlab, or R
  • (default) Test your local physician notes without retraining
  • (optional) Retrain model using local physician note samples, publications, and medical dictionaries.  

Compare and classify medical text

  • Generate feature set of lexical properties, medical concept codes, and human defined rules
  • Compare lexical properties of public and private text sources
  • Distinguish (classify) private patient data from coded medical concepts and commonly used words

How-To Guide


Installation
Train and test models
Scrub physician notes



Error rendering macro 'viewdoc'

com.atlassian.confluence.macro.MacroExecutionException: com.atlassian.confluence.macro.MacroExecutionException: The viewfile macro is unable to locate the attachment "scrubber-3.x-runtime-guide.doc" on this page

Scrubber Property KEY = VALUE



(Configure) scrubber.properties

  • No labels