... | ... | @@ -22,12 +22,16 @@ Le jeudi à 15h en 26-00/536. |
|
|
<td>[On Chomsky and the Two Cultures of Statistical Learning](http://norvig.com/chomsky.html)</td>
|
|
|
</tr><tr>
|
|
|
<td>Bruno</td>
|
|
|
<td><em>Tuto</em> <strong>Word Embeddings</strong></td>
|
|
|
<td><em>Tuto</em> <strong>[Word Embeddings](uploads/88a5bfd9a9c459accd00954dd3dced71/Word_embeddings.pdf)</strong></td>
|
|
|
<td></td>
|
|
|
</tr><tr>
|
|
|
<td>Edouard</td>
|
|
|
<td><em>Tuto</em> <strong>[Language model, LSTM et Subwords](uploads/5db67569d609d3c0d73189bcfd14b96d/Tuto_NLP_LSTM.pdf)</strong></td>
|
|
|
<td></td>
|
|
|
<td>
|
|
|
<strong>[LSTM](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.676.4320&rep=rep1&type=pdf)</strong>: [A search space odyssey](https://arxiv.org/pdf/1503.04069), [Capacity and trainability in recurrent neural networks](https://arxiv.org/pdf/1611.09913), [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/)<br />
|
|
|
<strong>Tips and tricks</strong>: [On the State of the Art of Evaluation in Neural Language Models](https://arxiv.org/pdf/1707.05589), [Regularizing and Optimizing LSTM Language Models](https://arxiv.org/pdf/1708.02182)<br />
|
|
|
<strong>Subwords</strong>: [BPE](https://arxiv.org/pdf/1508.07909) ([Wikipédia](https://en.wikipedia.org/wiki/Byte_pair_encoding), [implémentation](https://github.com/glample/fastBPE)), [WordPiece](https://ai.google/research/pubs/pub37842.pdf) ([implémentation](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/pytorch_pretrained_bert/tokenization.py)), [Subword](https://arxiv.org/pdf/1804.10959) ([implémentation](https://github.com/google/sentencepiece))
|
|
|
</td>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
<td rowspan="2">2019-06-13</td>
|
... | ... | @@ -187,7 +191,7 @@ Le jeudi à 15h en 26-00/536. |
|
|
<td>[encoder-decoder](https://arxiv.org/pdf/1406.1078.pdf), [attention](https://arxiv.org/pdf/1409.0473.pdf), [attention dot product](https://arxiv.org/pdf/1508.04025.pdf), [end-to-end memnet](https://papers.nips.cc/paper/5846-end-to-end-memory-networks.pdf), [attention is all you need](https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf)</td>
|
|
|
</tr>
|
|
|
<tr>
|
|
|
<td>2018-12-06</td>
|
|
|
~ <td>2018-12-06</td>
|
|
|
<td>Clara</td>
|
|
|
<td><strong>[Réseaux Hiérarchiques Attentifs pour la Représentation de Carrières Professionnelles](https://docs.google.com/presentation/d/1isXtCoMP5POA7Bfd0vYWOqQ5S8-UUSYqB4iIgjhFtIA/edit?usp=sharing&ts=5c092c6e)</strong></td>
|
|
|
<td></td>
|
... | ... | |