What is Retrieval Augmented Generation?

Large Language Models (LLMs) have contributed to advancing the domain of natural language processing (NLP), yet an existing gap persists in contextual understanding. LLMs can sometimes produce inaccurate or unreliable responses, a phenomenon known as “hallucinations.”  For instance, with ChatGPT, the occurrence of hallucinations is approximated…

Complex, unfamiliar sentences make the brain’s language network work harder

With help from an artificial language network, MIT neuroscientists have discovered what kind of sentences are most likely to fire up the brain’s key language processing centers.

The new study reveals that sentences that are more complex, either because of unusual grammar or unexpected meaning, generate stronger responses in these language processing centers. Sentences that are very straightforward barely engage these regions, and nonsensical sequences of words don’t do much for them either.

For example, the researchers found this brain network was most active when reading unusual sentences such as “Buy sell signals remains a particular,” taken from a publicly available language dataset called C4. However, it went quiet when reading something very straightforward, such as “We were sitting on the couch.”

“The input has to be language-like enough to engage the system,” says Evelina Fedorenko, Associate Professor of Neuroscience at MIT and a member of MIT’s McGovern Institute for Brain Research. “And then within that space, if things are really easy to process, then you don’t have much of a response. But if things get difficult, or surprising, if there’s an unusual construction or an unusual set of words that you’re maybe not very familiar with, then the network has to work harder.”

Fedorenko is the senior author of the study, which appears today in Nature Human Behavior. MIT graduate student Greta Tuckute is the lead author of the paper.

Processing language

In this study, the researchers focused on language-processing regions found in the left hemisphere of the brain, which includes Broca’s area as well as other parts of the left frontal and temporal lobes of the brain.

“This language network is highly selective to language, but it’s been harder to actually figure out what is going on in these language regions,” Tuckute says. “We wanted to discover what kinds of sentences, what kinds of linguistic input, drive the left hemisphere language network.”

The researchers began by compiling a set of 1,000 sentences taken from a wide variety of sources — fiction, transcriptions of spoken words, web text, and scientific articles, among many others.

Five human participants read each of the sentences while the researchers measured their language network activity using functional magnetic resonance imaging (fMRI). The researchers then fed those same 1,000 sentences into a large language model — a model similar to ChatGPT, which learns to generate and understand language from predicting the next word in huge amounts of text — and measured the activation patterns of the model in response to each sentence.

Once they had all of those data, the researchers trained a mapping model, known as an “encoding model,” which relates the activation patterns seen in the human brain with those observed in the artificial language model. Once trained, the model could predict how the human language network would respond to any new sentence based on how the artificial language network responded to these 1,000 sentences.

The researchers then used the encoding model to identify 500 new sentences that would generate maximal activity in the human brain (the “drive” sentences), as well as sentences that would elicit minimal activity in the brain’s language network (the “suppress” sentences).

In a group of three new human participants, the researchers found these new sentences did indeed drive and suppress brain activity as predicted.

“This ‘closed-loop’ modulation of brain activity during language processing is novel,” Tuckute says. “Our study shows that the model we’re using (that maps between language-model activations and brain responses) is accurate enough to do this. This is the first demonstration of this approach in brain areas implicated in higher-level cognition, such as the language network.”

Linguistic complexity

To figure out what made certain sentences drive activity more than others, the researchers analyzed the sentences based on 11 different linguistic properties, including grammaticality, plausibility, emotional valence (positive or negative), and how easy it is to visualize the sentence content.

For each of those properties, the researchers asked participants from crowd-sourcing platforms to rate the sentences. They also used a computational technique to quantify each sentence’s “surprisal,” or how uncommon it is compared to other sentences.

This analysis revealed that sentences with higher surprisal generate higher responses in the brain. This is consistent with previous studies showing people have more difficulty processing sentences with higher surprisal, the researchers say.

Another linguistic property that correlated with the language network’s responses was linguistic complexity, which is measured by how much a sentence adheres to the rules of English grammar and how plausible it is, meaning how much sense the content makes, apart from the grammar.

Sentences at either end of the spectrum — either extremely simple, or so complex that they make no sense at all — evoked very little activation in the language network. The largest responses came from sentences that make some sense but require work to figure them out, such as “Jiffy Lube of — of therapies, yes,” which came from the Corpus of Contemporary American English dataset.

“We found that the sentences that elicit the highest brain response have a weird grammatical thing and/or a weird meaning,” Fedorenko says. “There’s something slightly unusual about these sentences.”

The researchers now plan to see if they can extend these findings in speakers of languages other than English. They also hope to explore what type of stimuli may activate language processing regions in the brain’s right hemisphere.

The research was funded by an Amazon Fellowship from the Science Hub, an International Doctoral Fellowship from the American Association of University Women, the MIT-IBM Watson AI Lab, the National Institutes of Health, the McGovern Institute, the Simons Center for the Social Brain, and MIT’s Department of Brain and Cognitive Sciences.

Building technology that empowers city residents

Kwesi Afrifa came to MIT from his hometown of Accra, Ghana, in 2020 to pursue an interdisciplinary major in urban planning and computer science. Growing up amid the many moving parts of a large, densely populated city, he had often observed aspects of urban life that could be made more efficient. He decided to apply his interest in computing and coding to address these problems by creating software tools for city planners.

Now a senior, Afrifa works at the City Form Lab led by Andres Sevstuk, collaborating on an open-source, Python-based tool that allows researchers and policymakers to analyze pedestrians’ behaviors. The package, which launches next month, will make it more feasible for researchers and city planners to investigate how changes to a city’s structural characteristics impact walkability and the pedestrian experience.

During his first two years at MIT, Afrifa worked in the Civic Data Design Lab led by Associate Professor Sarah Williams, where he helped build sensing tools and created an online portal for people living in Kibera, Nairobi, to access the internet and participate in survey research.

After graduation, he will go on to work as a software engineer at a startup in New York. After several years, he hopes to start his own company, building urban data tools for integration into mapping and location-based software applications.

“I see it as my duty to make city systems more efficient, deepen the connection between residents and their communities, and make existing in them better for everyone, including groups which have often been marginalized,” he says.

“Cities are special places”

Afrifa believes that in urban settings, technology has a unique power to both accelerate development and empower citizens.

He witnessed such unifying power in high school, when he created the website ghanabills.com, which aggregated bills of parliament in Ghana, providing easy access to this information as well as a place for people to engage in discussion on the bills. He describes the effect of this technology as a “democratizing force.”

Afrifa also explored the connection between cities and community as an executive member of Code for Good, a program that connects MIT students interested in software with nonprofits throughout the Boston area. He served as a mentor for students and worked on finding nonprofits to match them up with.

Language and visibility

Sharing African languages and cultures is also important to Afrifa. In his first two years at MIT, he and other African students across the country started the Mandla app, which he describes as a Duolingo for African languages. It had gamified lessons, voice translations, and other interactive features for learning. “We wanted to solve the problem of language revitalization and bring African languages to the broader diaspora,” he says. At its peak a year ago, the app had 50,000 daily active users.

Although the Mandla App was discontinued due to lack of funding, Afrifa has found other ways to promote African culture at MIT. He is currently collaborating with architecture graduate students TJ Bayowa and Courage Kpodo on a “A Tale of Two Coasts,” an upcoming short film and multimedia installation that delves into the intricate connections between perceptions of African art and identity spanning two coasts of the Atlantic Ocean. This ongoing collaboration, which Afrifa says is still taking shape, is something he hopes to expand beyond MIT.

Discovering arts

As a child, Afrifa enjoyed writing poetry. Growing up with parents who loved literature, Afrifa was encouraged to become involved with the theater and art scene of Accra. He didn’t expect to continue this interest at MIT, but then he discovered the Black Theater Guild (BTG).

The theater group had been active at MIT from the 1990s to around 2005. It was revived by Afrifa in his sophomore year when Professor Jay Scheib, head of Music and Theater Arts at MIT, encouraged him to write, direct, and produce more of his work after his final project for 21M.710 (Script Analysis), a dramaturgy class taught by Scheib.

Since then, the BTG has held two productions in the past two years: “Nkrumah’s Last Day,” in spring 2022, and “Shooting the Sheriff,” in spring 2023, both of which were written and directed by Afrifa. “It’s been very rewarding to conceptualize ideas, write stories and have this amazing community of people come together and produce it,” he says.

When asked if he will continue to pursue theater post-grad, Afrifa says: “That’s 100 percent the goal.”

Artistic Devotion – A Journey Through Horror And Success With Red Candle Games

Scaring players has helped put the small team at Red Candle Games on the map. Its first two titles, Detention and Devotion, impressed and terrified fans with frighteningly strong horror chops, but the team is leaving that behind to tackle something completely different in Nine Sols. This side-scrolling action platformer blends sci-fi with Chinese mythology and is Red Candle’s most ambitious project yet. This significant step forward affords an opportunity to look back on the studio’s young history, so we spoke with co-founder Vincent Yang to learn about how a small band of gamers in Taiwan formed one of the industry’s most promising – and, at one time, controversial – indie studios.

Red Candle Games began in 2014 when designer Coffee Yao was developing a prototype for Detention. The side-scrolling horror game features two students exploring a haunted high school in 1960s Taiwan, incorporating local cultural references rarely seen in games. These elements drew the attention of Henry and Light Wang during an indie developer meetup in Taipei. According to Yang, Detention impressed the Wangs enough for them to join the project. As the game grew in scope, the three recruited help from mutual friends – Doy Chiang, Hans Chen, and Yang himself – bumping the yet-to-be-named team to six members by late 2015. After working remotely on Detention for a period, the group decided a physical office was necessary, which led to the official formation of Red Candle Games.

Detention was released on Steam in January 2017 and was well-received by critics and players. The game has since been ported to multiple platforms and even adapted into a live-action film in 2019 (it premiered in the U.S. in 2021). Additionally, Netflix and Taiwan’s Public Television Service released a Detention TV show in 2020 – though with a different story from the movie.

Devotion

Red Candle followed this success with Devotion, an atmospheric first-person horror game set within a haunted apartment. Moving from 2D to 3D development is a big design shift, but Yang believes regularly changing gears like this is necessary to maintain fan interest.

“Because in this ever-changing and competitive gaming market, we often find it difficult to impress the players by applying the same techniques repeatedly, which pushes us to keep delivering new and innovative ideas,” Yang says. “This, in turn, changes the way we make games.”

After over two years of work, Devotion arrived on Steam in February 2019 to heaps of praise from the lucky players who managed to get their hands on it. But only two days after its launch, the game riled up controversy among Chinese players thanks to a piece of in-game art mocking China’s president, Xi Jinping, by calling him Winnie the Pooh, referencing an internet meme that unfavorably compares the leader’s appearance to the character. China and Taiwan have a tense relationship, so the resulting backlash from the country’s players and the government was severe enough to force Red Candle to pull Devotion from sale. The game’s Chinese publisher even lost its business license.

Red Candle Games’ founders, (left to right): Doy Chiang, Hans Chen, Coffee Yao, Vincent Yang, Henry Wang, and Light Wang

To the dismay of horror enthusiasts, Devotion remained delisted for over two years. Inklings of hope would occasionally arise, such as PC storefront GOG announcing plans to re-sell the game before it immediately back-peddled due to vaguely defined player complaints. Fortunately, Red Candle finally began re-selling Devotion directly through its website in March 2021, bringing its tumultuous journey to an end. Red Candle could move forward. “As a team, we felt relieved, as if a heavy weight was lifted from our shoulders,” says Yang.

Moving forward meant stepping away from horror to do something different. Enter Nine Sols, which Red Candle started working on a few months before Devotion’s original launch. Yang says the game originates from a concept he envisioned years prior based on the story of Hou Yi. In Chinese mythology, Hou Yi is a legendary archer who used his bow to rescue the country from plagues and saved the moon during an eclipse, among other supernatural feats. Yang proposed reworking this story into a science fiction tale, an idea that intrigued his teammates.

Nine Sols unfolds in the realm of New Kunlun, a sanctuary for humans created by nine god-like rulers. However, the population is unaware of New Kunlun’s sinister true purpose. That changes when a child awakens Yi, a cat-like hero that history has forgotten. Yi embarks on a quest to slay the nine rulers and free New Kunlun. The adventure sports a manga-inspired hand-drawn art direction blending cyberpunk/sci-fi with Chinese fantasy and Taoism. Gameplaywise, Nine Sols is a fast-paced 2D Metroidvania with Souls-like combat, as players hack apart enemies using Yi’s sword and parry attacks in a manner directly influenced by Sekiro: Shadows Die Twice.

The project is a gamble, as the team leaves its comfort zone to make its first action platformer. Yang says with its art assets, gameplay mechanics, levels, animations, and 12-15 hour length, Nine Sols is the studio’s biggest title yet. But the team needed to do something different to avoid feeling burned out on horror while also taking advantage of the flexibility its team size affords.

Red Candle Games’ upcoming Nine Sols pays homage to the Chinese myth of Hou Yi, and features a manga-inspired art style

“Being an indie developer, in a way, makes us more versatile and gives us more room to be creative, meaning we have the privilege to experiment on different topics and gameplay,” Yang says. “It’s then reasonable that we want to leverage that versatility and explore other possibilities wherever we have the chance.”

Nine Sols is Red Candle’s first action game, but it’s leveraging its narrative chops to craft what Yang hopes will be a compelling story filled with rich lore to create what he describes as “a new kind of action platformer” – one with equal emphasis on engaging gameplay and storytelling.

Detention

Nine Sols also marks the team’s most serious attempt at penetrating the Western market. Detention and Devotion, despite their success, still appeal to relatively niche audiences. On the other hand, Nine Sols combines two very popular subgenres, and Yang hopes that results in the game making the biggest splash possible overseas and putting the studio on more people’s radars.

Nine Sols is currently slated to launch in Q1 2024, but curious players can play a free Steam demo. Depending on its success, the title could be a launch pad for other experiences that don’t focus exclusively on frightening players.

“If given the opportunity, I think we’d love to explore other genres in the future,” says Yang. “Deep down, we are simply a group of game creators who love all kinds of games. So as long as we ensure the product we produce will have guaranteed quality and doesn’t disappoint our current and future supporters, I find no reasons for us to be locked into the horror genre.”

As for Red Candle’s future, Yang believes maintaining its small size has its benefits. Since Devotion’s launch, only three new employees have joined, while one has left, meaning the studio has grown from 11 to 13 full-time staff members. While he’s happy with the advantages of running a small team – smoother communication and more room to experiment – he’s not opposed to expanding the studio’s size if needed. “I envision us as a resilient team that can embrace and adapts to any possible changes regardless of our size,” Yang says.

Nine Sols is the biggest test yet for the studio, eschewing almost everything that brought it success in an attempt to broaden its appeal by tackling a competitive genre. But Red Candle Games has proven its ability, resiliency, and versatility over its relatively young existence. We’re excited to see how it continues to delight – or terrify – its fans going forward.

This article originally appeared in Issue 356 of Game Informer

Ubisoft Corrects Star Wars Outlaws Releasing In Late 2024 Back To General 2024

Update: Earlier today, a Disney Parks blog pointed to Star Wars Outlaws releasing in late 2024, but the term “late” has now been removed from the post so that is now reads just 2024. Ubisoft also confirmed to Kotaku that, “the timing was incorrect and has since been amended.” So, Star Wars Outlaws’ release window has moved from late 2024 back to some general time this year.

Original story:

Star Wars Outlaws, Ubisoft’s upcoming open-world Star Wars game, has enjoyed a vague 2024 release window since it was revealed at an Xbox Showcase last year. However, a recent post on Disney’s official park blog has given the game a more specific window. According to the blog, Star Wars Outlaws is releasing late 2024. The blog post, which is a collection of 24 exciting Disney-things happening in 2024 reads, “And for Star Wars fans – Star Wars Outlaws, the open-world Star Wars game is set to release late this year.”

Star Wars Outlaws follows an original Star Wars character named Kay Vess and seeks to explore a character archetype of the Star Wars universe we haven’t spent much time with in video games: the Han Solo-inspired scoundrel. In the game, Vess (who has no Jedi abilities as far as we know) must survive by her wits and her ability to talk (or blast) her way through difficult situations.

For more on Star Wars Outlaws, you can read our interview with the game’s narrative director, Navid Khavari, where we covered topics like whether or not the game will have an overlap with Shadows of the Empire and if the team has been communicating with Respawn regarding its Jedi series of games. The game is planned for release on PlayStation 5, Xbox Series X/S, and PC.

Pioneering ASD Diagnosis Through AI and Retinal Imaging

In the realm of healthcare, particularly in the diagnosis of Autism Spectrum Disorder (ASD), a groundbreaking study has emerged. Traditionally, diagnosing ASD has been a domain reliant on the expertise of specialized professionals, a process that is often exhaustive and not universally accessible. This has led…