Bezen

Your daily digest

Research papers, made simple to understand

The AI and machine learning papers people are reading most, each one rewritten in plain words: the problem, the idea, what was found and why it matters.

Language Models & Architectures8 papers, newest first

    • Language Models & Architectures

    Two efficient models learn high-quality word vectors from huge text corpora

    This paper introduces the Continuous Bag-of-Words and Skip-gram models, which efficiently learn word embeddings from billions of words, achieving state-of-the-art accuracy on semantic and syntactic similarity tasks with much less computation than prior neural network models.

    Paper: Efficient Estimation of Word Representations in Vector Space · Tomas Mikolov, Kai Chen, Greg Corrado and 1 other

    Classic · 201318 Oct 20265 min readRead the article

    • Language Models & Architectures

    Neural machine translation improves by learning soft alignments during decoding

    This paper introduces a neural machine translation model that jointly learns to align source words with target words using a soft attention mechanism, achieving translation quality comparable to phrase-based systems on English-to-French tasks and better handling long sentences.

    Paper: Neural Machine Translation by Jointly Learning to Align and Translate · Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio

    Classic · 201418 Oct 20267 min readRead the article

    • Language Models & Architectures

    Deep LSTM networks map reversed input sequences to fixed vectors for strong machine translation

    This paper introduces a sequence-to-sequence model using two deep LSTMs—one encoding reversed source sentences into fixed vectors, the other decoding target sentences—achieving a BLEU score of 34.8 on WMT'14 English-French, outperforming phrase-based SMT baselines.

    Paper: Sequence to Sequence Learning with Neural Networks · Ilya Sutskever, Oriol Vinyals, Quoc V. Le

    Classic · 201418 Oct 20266 min readRead the article

    • Language Models & Architectures

    Transformer replaces recurrence with multi-head self-attention for faster, better translation

    The Transformer model uses only attention mechanisms, no recurrence or convolution, achieving state-of-the-art BLEU scores on WMT 2014 English-German (28.4) and English-French (41.8) tasks with significantly less training time and cost.

    Paper: Attention Is All You Need · Ashish Vaswani, Noam Shazeer, Niki Parmar and 5 others

    Classic · 201717 Oct 20266 min readRead the article

    • Language Models & Architectures

    BERT pre-trains deep bidirectional Transformers for versatile NLP tasks

    BERT introduces masked language modeling and next sentence prediction to pre-train deep bidirectional Transformer encoders, achieving state-of-the-art results on 11 NLP benchmarks including GLUE, SQuAD v1.1 and v2.0, with minimal task-specific changes.

    Paper: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding · Jacob Devlin, Ming-Wei Chang, Kenton Lee and 1 other

    Classic · 201817 Oct 20266 min readRead the article

    • Language Models & Architectures

    Scaling up GPT-3 enables strong few-shot NLP task performance without fine-tuning

    GPT-3, a 175B parameter autoregressive language model, achieves competitive few-shot learning on diverse NLP tasks by conditioning on task demonstrations in text, without any gradient updates or fine-tuning, showing smooth performance gains with scale.

    Paper: Language Models are Few-Shot Learners · Tom B. Brown, Benjamin Mann, Nick Ryder and 28 others

    Classic · 202017 Oct 20266 min readRead the article

    • Language Models & Architectures
    • Training & Efficiency

    ALoDLM improves diffusion language models with token-adaptive recurrent refinement

    ALoDLM introduces token-adaptive latent recurrence within diffusion language models, allowing tokens to commit at different depths based on prediction difficulty. Trained at 1.7B and 8B scales, it outperforms prior diffusion and autoregressive models across 11 benchmarks while maintaining fast parallel decoding.

    Paper: ALoDLM: Adaptively Looped Diffusion Language Models · Liancheng Fang, Zhuowei Li, Youngeun Kim and 10 others

    60 upvotes on Hugging Face6 Oct 20268 min readRead the article

    • LLM Reasoning
    • Language Models & Architectures

    Protein folding training improves broad reasoning in language models

    Training a language model on protein folding-derived question-answer data and 3D geometry improves its performance on ten diverse reasoning benchmarks by 3.23 percentage points, showing that structured scientific data can enhance general reasoning.

    Paper: Does Learning Protein Folding Generalize to Broader Reasoning? · Yong Liu, Zhanpeng Shi, Yizhou Dang and 4 others

    105 upvotes on Hugging Face5 Oct 20267 min readRead the article