Class DBBulkLoader


  • public class DBBulkLoader
    extends java.lang.Object
    This class reads through two files (data and metadata), tokenizing metadata for use as a label vector.
    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected static java.util.logging.Logger logger  
    • Constructor Summary

      Constructors 
      Constructor Description
      DBBulkLoader()  
    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static void generateStoplist​(SimpleTokenizer prunedTokenizer, NGramPreprocessor preprocessor)
      Read the data from inputFiles, then write all the words that do not occur pruneCount.value times or more to the pruned word file.
      static void main​(java.lang.String[] args)  
      static void writeInstanceList​(java.util.ArrayList<Pipe> pipes)  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • logger

        protected static java.util.logging.Logger logger
    • Constructor Detail

      • DBBulkLoader

        public DBBulkLoader()
    • Method Detail

      • generateStoplist

        public static void generateStoplist​(SimpleTokenizer prunedTokenizer,
                                            NGramPreprocessor preprocessor)
                                     throws java.io.IOException
        Read the data from inputFiles, then write all the words that do not occur pruneCount.value times or more to the pruned word file.
        Parameters:
        prunedTokenizer - the tokenizer that will be used to write instances
        Throws:
        java.io.IOException
      • writeInstanceList

        public static void writeInstanceList​(java.util.ArrayList<Pipe> pipes)
                                      throws java.lang.Exception
        Throws:
        java.lang.Exception
      • main

        public static void main​(java.lang.String[] args)
                         throws java.lang.Exception
        Throws:
        java.lang.Exception