#A Collection of NLP notes
##N-grams
###Calculating unigram probabilities:
P( wi ) = count ( wi ) ) / count ( total number of words )
In english..
Bo Wang bwang482
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import numpy as np | |
| from scipy.sparse import csr_matrix | |
| import torch | |
| __author__ = 'Andrea Esuli' | |
| Acsr = csr_matrix([[1, 2, 0], [0, 0, 3], [4, 0, 5]]) | |
| print('Acsr',Acsr) | |
| Acoo = Acsr.tocoo() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Author: Jake VanderPlas | |
| # LICENSE: MIT | |
| from __future__ import division | |
| import numpy as np | |
| def convolution_matrix(x, N=None, mode='full'): | |
| """Compute the Convolution Matrix |
lampts
/ gensim2projector_tf.py
Last active
December 7, 2020 22:37
how to convert/port gensim word2vec to tensorflow projector board.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # required tensorflow 0.12 | |
| # required gensim 0.13.3+ for new api model.wv.index2word or just use model.index2word | |
| from gensim.models import Word2Vec | |
| import tensorflow as tf | |
| from tensorflow.contrib.tensorboard.plugins import projector | |
| # loading your gensim | |
| model = Word2Vec.load("YOUR-MODEL") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """Information Retrieval metrics | |
| Useful Resources: | |
| http://www.cs.utexas.edu/~mooney/ir-course/slides/Evaluation.ppt | |
| http://www.nii.ac.jp/TechReports/05-014E.pdf | |
| http://www.stanford.edu/class/cs276/handouts/EvaluationNew-handout-6-per.pdf | |
| http://hal.archives-ouvertes.fr/docs/00/72/67/60/PDF/07-busa-fekete.pdf | |
| Learning to Rank for Information Retrieval (Tie-Yan Liu) | |
| """ | |
| import numpy as np |