About me

My name is Feng, I am passionate about uncovering the fundamental principles of biological intelligence and exploring how they can be replicated through artificial systems. My current work focuses on Natural Language Processing, Free Energy Principle, and Active Inference—areas that offer profound insights into intelligence. As this dynamic field evolves, my focus is likely to shift toward whichever area promises the highest long-term impact in the quest to understand intelligence at a deep level.

Currently serve as a Software Engineer at AI Rudder, a startup based in San Mateo, California, where we empower large language models (LLMs) and integrate cutting-edge Text-to-Speech (TTS) and Automatic Speech Recognition (ASR) technologies. Our mission is to develop innovative voice bots for phone interactions, pushing the boundaries of conversational AI. I earned my master’s degree in Computer Science from the University of Illinois at Urbana-Champaign in Winter 2021, following my work as a Research Intern at Westlake University under the guidance of Prof. Lanzhenzhong . Before this, I obtained my bachelor’s degree in Computer Science from Ohio State University in 2020.

Outside of work, I enjoy working out, reading, and exploring fields like cognitive science, neuroscience, psychology, and physics. At my core, I aim to minimize the free energy of my internal world model—seeking the simplest way to understand and express the world around me. These activities continuously expand my horizons and fuel my curiosity.

Publication

  • Toward the Limitation of Code-Switching in Cross-Lingual Transfer

    Y Feng, F Li, P Koehn EMNLP

    Multilingual pretrained models have shown strong cross-lingual transfer ability. Some works used code-switching sentences, which consist of tokens from multiple languages, to enhance the cross-lingual representation further, and have shown success in many zero-shot cross-lingual tasks. However, code-switched tokens are likely to cause grammatical incoherence in newly substituted sentences, and negatively affect the performance on token-sensitive tasks, such as Part-of-Speech (POS) tagging and Named-Entity-Recognition (NER). This paper mitigates the limitation of the code-switching method by not only making the token replacement but considering the similarity between the context and the switched tokens so that the newly substituted sentences are grammatically consistent during both training and inference. We conduct experiments on cross-lingual POS and NER over 30+ languages, and demonstrate the effectiveness of our method by outperforming the mBERT by 0.95 and original code-switching method by 1.67 on F1 scores.

    Paper Code

    December 2022

  • Learn To Remember: Transformer With Recurrent Memory For Document-Level Machine Translation

    Y Feng, F Li, Z Song, B Zheng, P Koehn NAACL

    The Transformer architecture has led to significant gains in machine translation. However, most studies focus on only sentence-level translation without considering the context dependency within documents, leading to the inadequacy of document-level coherence. Some recent research tried to mitigate this issue by introducing an additional context encoder or translating with multiple sentences or even the entire document. Such methods may lose the information on the target side or have an increasing computational complexity as documents get longer. To address such problems, we introduce a recurrent memory unit to the vanilla Transformer, which supports the information exchange between the sentence and previous context. The memory unit is recurrently updated by acquiring information from sentences, and passing the aggregated knowledge back to subsequent sentence states. We follow a two-stage training strategy, in which the model is first trained at the sentence level and then finetuned for document-level translation. We conduct experiments on three popular datasets for document-level machine translation and our model has an average improvement of 0.91 s-BLEU over the sentence-level baseline. We also achieve state-of-the-art results on TED and News, outperforming the previous work by 0.36 s-BLEU and 1.49 d-BLEU on average.

    Paper Code

    July 2022

What's New

Learn To Remember: Transformer With Recurrent Memory For Document-Level Machine Translation

The Transformer architecture has led to significant gains in machine translation. However, most studies focus on only sentence-level translation without considering the context dependency within documents, leading to the inadequacy of document-level coherence. Some recent research tried to mitigate this issue by introducing an additional context encoder or translating with multiple sentences or even the entire document. Such methods may lose the information on the target side or have an increasing computational complexity as documents get longer. To address such problems, we introduce a recurrent memory unit to the vanilla Transformer, which supports the information exchange between the sentence and previous context. The memory unit is recurrently updated by acquiring information from sentences, and passing the aggregated knowledge back to subsequent sentence states. We follow a two-stage training strategy, in which the model is first trained at the sentence level and then finetuned for document-level translation. We conduct experiments on three popular datasets for document-level machine translation and our model has an average improvement of 0.91 s-BLEU over the sentence-level baseline. We also achieve state-of-the-art results on TED and News, outperforming the previous work by 0.36 s-BLEU and 1.49 d-BLEU on average.

Paper Code