Ray RayOnFire

Interest in machine learning, especially NLP and data mining.

RayOnFire / fastai_anon_series.py

Created November 16, 2019 16:11

Fastai处理匿名序列

	from fastai.text import TextLMDataBunch, TextClasDataBunch, \
	language_model_learner, AWD_LSTM, LinearDecoder, RNNLearner, Tokenizer, SequentialRNN,\
	text_classifier_learner
	from torch import nn
	import torch
	from tqdm import tqdm_notebook as tqdm
	import numpy as np

	tokenizer = Tokenizer(post_rules=[]) # 不要进行fastai自带的大小写转换

RayOnFire / install_lightgbm.sh

Created November 12, 2019 03:35

没有sudo的情况下安装GPU版本LightGBM

	conda install cmake libboost -y
	git clone https://github.com/KhronosGroup/OpenCL-Headers

	git clone --recursive https://github.com/microsoft/LightGBM
	cd LightGBM
	mkdir build ; cd build
	cmake -DOpenCL_LIBRARY=/usr/lib/x86_64-linux-gnu/libOpenCL.so.1 -DOpenCL_INCLUDE_DIR=../../OpenCL-Headers/ ..
	# if you have installed NVIDIA CUDA to a customized location, you should specify paths to OpenCL headers and library like the following:
	# cmake -DUSE_GPU=1 -DOpenCL_LIBRARY=/usr/local/cuda/lib64/libOpenCL.so -DOpenCL_INCLUDE_DIR=/usr/local/cuda/include/ ..
	make -j$(nproc)

RayOnFire / snippet.py

Created September 20, 2018 03:25

[Pandas with json fields] #Python #Pandas

	def load_df(csv_path='../input/train.csv', nrows=None):
	JSON_COLUMNS = ['device', 'geoNetwork', 'totals', 'trafficSource']

	df = pd.read_csv(csv_path,
	converters={column: json.loads for column in JSON_COLUMNS},
	dtype={'fullVisitorId': 'str'}, # Important!!
	nrows=nrows)

	for column in JSON_COLUMNS:
	column_as_df = json_normalize(df[column].tolist()) # 这里tolist跟其他教程不一样，因为直接输入Series在这里不能解析，不知道是不是版本原因

RayOnFire / redirect.sh

Created June 26, 2018 06:14

[Shell 输出重定向]

nohup abc.sh > nohup.log 2>&1 &

RayOnFire / foreach.xml

Created June 15, 2018 03:30

[Java MyBatis foreach]#Java #MyBatis

	<foreach collection="organizations" item="item" index="index"
	open="(" separator="," close=")">#{item.id}</foreach>

RayOnFire / delete

Created June 12, 2018 02:57

[Git Add deleted files]

git add -u :/

RayOnFire / FileUtils.java

Created June 5, 2018 07:19

[Java File Utils]Java文件操作 #Java

	import org.springframework.core.io.ClassPathResource;

	import java.io.BufferedReader;
	import java.io.IOException;
	import java.io.InputStream;
	import java.io.InputStreamReader;

	public class FileUtils {

	public static String readFromClassResource(String filename) throws IOException {

RayOnFire / walk.py

Created June 5, 2018 06:43

[Python Walk File System]遍历文件系统 #Python

	import os

	for root, dirs, files in os.walk("/"):
	path = root.split(os.sep)
	print((len(path) - 1) * '---', os.path.basename(root))
	for file in files:
	absolute_filename = os.path.join(root, file)
	print(absolute_filename)

RayOnFire / sha1.py

Created June 5, 2018 06:42

[Python SHA1]Python SHA1用法 #Python

	import hashlib

	salt = "some_salt"
	sha1 = hashlib.sha1()
	sha1.update(bytes('test' + salt, 'utf8'))
	print(sha1.hexdigest())

RayOnFire / mysql.py

Created June 5, 2018 06:39

[Python MySQL]MySQL连接模版 #Python #MySQL

	import pymysql
	import traceback

	cnx = pymysql.connect(host='localhost', port=3306,
	user='root', password='', database='db',
	charset='utf8mb4',
	cursorclass=pymysql.cursors.DictCursor)

	with cnx.cursor() as cursor:
	try: