Moved to repository: https://github.com/Mailcloud/swift-serializer
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /** | |
| * Created with IntelliJ IDEA. | |
| * User: Alexander <estliberitas> Makarenko | |
| * Date: 04.01.13 | |
| * Time: 3:25 | |
| */ | |
| 'use strict'; | |
| var fs = require('fs') | |
| , net = require('net') |
Install ImageMagick for image conversion:
brew install imagemagick
Install tesseract for OCR:
brew install tesseract --all-languages
Or install without --all-languages and install them manually as needed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| curl -s -XPOST localhost:9200/_bulk?pretty=true --data-binary ' | |
| { "index" : { "_index" : "parent_child", "_type" : "store", "_id" : "store1" } } | |
| { "name" : "auchan", "owner" : "chris" } | |
| { "index" : { "_index" : "parent_child", "_type" : "department", "_id" : "department1", "parent" : "store1" } } | |
| { "name" : "toys", "numberOfProducts" : 150 } | |
| { "index" : { "_index" : "parent_child", "_type" : "product", "_id" : "product1", "parent" : "department1", "routing" : "store1" } } | |
| { "name" : "gun", "trademark" : "tiger", "price" : 9, "store_id" : "store1" } | |
| ' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # herein we backup our indexes! this script should run at like 6pm or something, after logstash | |
| # rotates to a new ES index and theres no new data coming in to the old one. we grab metadatas, | |
| # compress the data files, create a restore script, and push it all up to S3. | |
| TODAY=`date +"%Y.%m.%d"` | |
| INDEXNAME="logstash-$TODAY" # this had better match the index name in ES | |
| INDEXDIR="/usr/local/elasticsearch/data/logstash/nodes/0/indices/" | |
| BACKUPCMD="/usr/local/backupTools/s3cmd --config=/usr/local/backupTools/s3cfg put" | |
| BACKUPDIR="/mnt/es-backups/" | |
| YEARMONTH=`date +"%Y-%m"` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # TO_FOLDER=/something | |
| # FROM=/your-es-installation | |
| DATE=`date +%Y-%m-%d_%H-%M` | |
| TO=$TO_FOLDER/$DATE/ | |
| echo "rsync from $FROM to $TO" | |
| # the first times rsync can take a bit long - do not disable flusing | |
| rsync -a $FROM $TO | |
| # now disable flushing and do one manual flushing |