Artem Lukanin avlukanin
jettro
/ TryHyphenation.java
Last active
November 3, 2019 21:13
Class to play around with Lucene HyphenationCompoundWordTokenFilter without booting an elasticsearch cluster. Want to see what happens to the hyphens and the found tokens with different setups and strings.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package com.klm.mysearch; | |
| import org.apache.lucene.analysis.CharArraySet; | |
| import org.apache.lucene.analysis.TokenStream; | |
| import org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter; | |
| import org.apache.lucene.analysis.compound.hyphenation.Hyphenation; | |
| import org.apache.lucene.analysis.compound.hyphenation.HyphenationTree; | |
| import org.apache.lucene.analysis.standard.StandardTokenizer; | |
| import org.junit.Ignore; | |
| import org.slf4j.Logger; |
vsdev1
/ LanguageDetectionComponent
Created
January 13, 2014 09:26
Solr search component for language detection
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /** | |
| * Solr search component for language detection. It has the following configuration parameters (are configured in solrconfig.xml): | |
| * <ul> | |
| * <li>list of parameters language: supported languages</li> | |
| * <li>languageDetectionProbabilityThreshold: the minimum detection probability for which the language detection is considered reliable (default is 0.09)</li> | |
| * </ul> | |
| */ | |
| public class LanguageDetectionComponent extends SearchComponent { | |
| public static final String LANGUAGE_REQUEST_PARAM_NAME = "language"; |
vsdev1
/ Common terms query parser plugin for Apache Solr
Last active
January 2, 2016 20:29
Common terms query parser plugin for Apache Solr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /** | |
| * Solr query parser login for the lucene common terms query. Must be used like q={!commonTermsQueryParser}WeightThe weight... | |
| * where commonTermsQueryParser is configured in solrconfig.xml. | |
| * | |
| * The query field must be given as query parameter "qf". The maxTermFrequency can be configured in solrconfig.xml | |
| * (default is 0.01). The query is executed against the field given in the request parameter qf while | |
| * using the query analyzers of this field. | |
| * | |
| * see http://java.dzone.com/articles/create-custom-solr-queryparser | |
| */ |
shlomibabluki
/ facebook_ner.py
Created
August 25, 2013 10:35
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # coding=UTF-8 | |
| from __future__ import division | |
| import nltk | |
| import re | |
| import requests | |
| # Add your freebase key here | |
| # If you don't have one, register at https://code.google.com/apis/console | |
| FREEBASE_KEY = "" |
shlomibabluki
/ freebase_ner.py
Created
August 25, 2013 10:34
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # coding=UTF-8 | |
| from __future__ import division | |
| import nltk | |
| import re | |
| import requests | |
| # Add your freebase key here | |
| # If you don't have one, register at https://code.google.com/apis/console | |
| FREEBASE_KEY = "" |
JnBrymn
/ SearchAsYouType.html
Created
June 8, 2013 04:00
This is a demo search UI for demonstrating Search As You Type with Solr.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <!doctype html> | |
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8" /> | |
| <title>search as you type</title> | |
| <link rel="stylesheet" href="http://code.jquery.com/ui/1.10.3/themes/smoothness/jquery-ui.css" /> | |
| <script src="http://code.jquery.com/jquery-1.9.1.js"></script> | |
| <script src="http://code.jquery.com/ui/1.10.3/jquery-ui.js"></script> | |
| <style> |