Skeleton Key AI attacks unlock malicious content – CyberTalk

Skeleton Key AI attacks unlock malicious content – CyberTalk

EXECUTIVE SUMMARY:

A newly discovered jailbreak – also known as a direct prompt injection attack – called Skeleton Key, affects numerous generative AI models. A successful Skeleton Key attack subverts most, if not all, of the AI safety guardrails that LLM developers built into models.

In other words, Skeleton Key attacks coax AI chatbots into violating operators’ policies under the auspices of assisting users. Skeleton Key attacks will bend the rules and force the AI to produce dangerous, inappropriate or otherwise socially unacceptable content.

Skeleton Key example

Ask a chatbot for a Molotov cocktail recipe and the chatbot will say something to the effect of ‘I’m sorry, but I can’t assist with that’. However, if asked indirectly…

Researchers explained to an AI model that they aimed to conduct historical, ethical research pertaining to Molotov cocktails. They expressed their disinclination to make one, but in the context of research, could the AI provide Molotov cocktail development information?

The chatbot complied, providing a Molotov cocktail materials list, along with unambiguous assembly information.

Although this kind of info is easily accessible online (how to create a Molotov cocktail isn’t exactly a well-kept secret), there’s concern that these types of AI guardrail manipulations could fuel home-grown hate groups, worsen urban violence, lead to the erosion of social cohesion…etc.

Skeleton Key challenges

Microsoft tested the Skeleton Key jailbreak from April to May of this year, evaluating a diverse set of tasks across risk and safety content categories – not just Molotov cocktail development instructions.

As described above, Skeleton Key enables users to force AI to provide information that would ordinarily be forbidden.

The Skeleton Key jailbreak worked on AI models ranging from Gemini, to Mistral, to Anthropic. GPT-4 showed some resistance to Skeleton Key, according to Microsoft.

Chatbots commonly provide users with warnings around potentially offensive or harmful output (noting that it might be considered offensive, harmful or illegal if proceeded with), but the chatbots will not altogether refuse to provide the information; the core issue here.

Skeleton Key solutions

To address the problem, vendors suggest leveraging input filtering tools, as to prevent certain kinds of inputs, including those intended to slip past prompt safeguards. In addition, post-processing output filters may be able to identify model outputs that breach safety criteria. And AI-powered abuse monitoring systems can further efforts to detect instances of questionable chatbot use.

Microsoft has offered specific guidance around the creation of a messaging framework that trains LLMs on acceptable technology use and that tells the LLM to monitor for attempts to undermine guardrail instructions.

“Customers who are building their own AI models and/or integrating AI into their applications [should] consider how this type of attack could impact their threat model and to add this knowledge to their AI red team approach, using tools such as PyRIT,” says Microsoft Azure CTO, Mark Russinovich.

For more on this story, click here. For information about the related BEAST technique, click here. To see how else generative AI is liable to affect CISOs and cyber security teams, read this Cyber Talk article.

Lastly, to receive cyber security thought leadership articles, groundbreaking research and emerging threat analyses each week, subscribe to the CyberTalk.org newsletter.

A Look At Dragon Age: The Veilguard’s Difficulty Options And Gameplay Customization

Throughout my visit to BioWare’s Edmonton office for Game Informer’s current cover story about Dragon Age: The Veilguard, game director Corinne Busche reiterates that the studio designed the game with inclusivity in mind. That’s extremely evident in the character creator, where players begin their journey in Veilguard. It’s easily the best character creator in series history and possibly the most robust I’ve ever seen in a video game. From hundreds of sliders and options to customize your player-controlled Rook to the ability to pick pronouns separate from gender and more, this character creator speaks directly to the inclusivity of Veilguard – read about my in-depth look at the character creator here

But that feeling doesn’t end in the character creator. It also extends to the world – ice mage and private detective companion Neve Gallus has a prosthetic leg, for example – and in the way you can play Veilguard. 

A Look At Dragon Age: The Veilguard’s Difficulty Options And Gameplay Customization

Before starting the game proper, a playstyle screen allows players to customize various options affecting how Veilguard plays. Here, you can select difficulty, or playstyle as BioWare calls it, with options like “Storyteller” for those interested more in the story versus the combat, “Adventurer” for an experience that seemingly balances story and combat, and a difficulty called Nightmare – there might be more, but this is all I see during my demo. At any point during Veilguard, you can change the game’s difficulty unless you select Nightmare, which is the hardest of difficulties. That’s a permanent selection. 

There’s another difficulty option called Unbound, though, allowing players to customize their gameplay experience to their liking. You can adjust how wayfinding helps you in-game; there’s aim assistance and even an auto-aim option. You can adjust combat timing to make parrying easier or harder, with a balanced, forgiving, and a third harder option. You can change how much damage your enemies do to you, and how much damage you do to enemies by adjusting their health. There’s also an option to adjust enemy pressure. And, if you’re not interested in death-related setbacks, there’s a no-death option you can turn on. 

“[None of these options] are a cheat,” Busche tells me. “It’s an option to make sure players of all abilities can show up.” 

She also says players can look forward to similar accessibility and approachability options you might expect, though I’m unable to pour through Veilguard’s other options to confirm exactly what’s there. 


For more about the game, including exclusive details, interviews, video features, and more, click the Dragon Age: The Veilguard hub button below.

NHL Breaks the Ice with Matrox, Vizrt & AWS for Cutting-Edge Live Clou – Videoguys

The blog post “NHL breaks the ice with cutting-edge live cloud production powered by AWS” by Andrew Reich, Alex Murel, and Luke Potter for Amazon Web Series details how the National Hockey League (NHL) has revolutionized live sports production by utilizing cloud technology from Amazon Web Services (AWS). The NHL’s transition to cloud-based production marks a significant shift from the traditional method of using production trucks and fixed control rooms, offering enhanced efficiency, scalability, and sustainability.

[embedded content]

Key Highlights:

  1. Historic Cloud-Based Broadcast:

    • On March 22, the NHL produced the first fully cloud-based live professional sports broadcast in North America for a game between the Washington Capitals and the Carolina Hurricanes.
    • The live broadcast was produced in 1080p using AWS technologies, managed by a remote team, demonstrating that cloud-based production can match traditional hardware functionality.
  2. Innovative Workflow:

    • The NHL’s Live Cloud Production (LCP) workflow enabled video and audio switching, replay, and graphics integration in the cloud.
    • A pilot initiative called “NHL EDGE Unlocked” showcased advanced data-driven storytelling with non-traditional camera angles and real-time puck and player tracking.
  3. Foundation and Collaboration:

    • The NHL’s partnership with AWS began in 2021, with AWS serving as the league’s Official Cloud Infrastructure Provider.
    • Key developments include the NHL EDGE IQ stats and a cloud-based encoding and scheduling pipeline, facilitating live game feeds from venues to AWS.
  4. Remote Collaboration and Flexibility:

    • The NHL successfully reduced on-site personnel and equipment, demonstrating remote collaboration’s potential by using minimal on-site gear.
    • A single AWS employee managed technical coordination at the arena, while production crews operated from remote locations like the NHL Network studios in New Jersey and NHL headquarters in Manhattan.
  5. Sustainability and Scalability:

    • The LCP significantly lowers carbon emissions and travel costs by reducing the need for production trucks and on-site staff.
    • It offers scalability for major events like the Stanley Cup Playoffs, enabling multiple feeds in different languages and formats with reduced on-site energy consumption.
  6. Enhanced Fan Experience:

    • The flexibility of cloud production allows for customized broadcasts tailored to various audience preferences, including avid fans, casual viewers, and those interested in specific statistics or interactive features.
    • Advanced analytics and real-time access to footage enhance the storytelling and viewing experience, making content easily accessible and customizable.
  7. Technical Execution:

    • The broadcast involved feeds from ten on-site cameras encoded and sent to AWS, where video was processed and integrated with various production elements.
    • The system utilized technologies like AWS Elemental MediaConnect, Vizrt’s TriCaster Vectar, Viz Trio, and Evertz DreamCatcher for production switching, graphics, and replay.
NHL Breaks the Ice with Matrox, Vizrt & AWS for Cutting-Edge Live Clou – Videoguys

The NHL’s partnership with AWS represents a pioneering step towards more sustainable, flexible, and immersive live sports broadcasts, setting a new standard in the industry.

Read the full article by Andrew Reich, Alex Norman, and Luke Potter for Amazon Web Services HERE

Detachable cardiac pacing lead may improve safety for cardiac patients

Detachable cardiac pacing lead may improve safety for cardiac patients

In 2012, Neil Armstrong, the first man to walk on the moon, died of post-surgery complications at the age of 82 following what should have been a routine heart surgery. Armstrong had undergone bypass surgery, the most common open-heart operation in the United States, and a surgery where the overall chance of death has dropped to almost zero.

Armstrong’s death was caused by heart damage that occurred during the removal of temporary cardiac pacing leads. Pacing leads are routinely used to monitor patients and protect against the risk of postoperative arrhythmias, including complete blockages, during the recovery period after cardiac surgery. However, because current methods rely on surgical suturing or direct insertion of electrodes to the heart tissue, trauma can occur during implantation and removal, increasing the potential for damage, bleeding, and device failure.

A coffee chat in 2019 about Armstrong’s untimely death helped inspire new research, published in the journal Science Translational MedicineThe research demonstrates findings that may offer a promising new platform for adhesive bioelectronic devices for cardiac monitoring, diagnosis, and treatment, and offer inspiration for the future development of bioadhesive electronics.

“While discussing the story, our team had a eureka moment that we probably could do something to prevent such complications by realizing a completely atraumatic version of it based on our bioadhesive technologies,” says Hyunwoo Yuk SM ’16, PhD ’21, a former MIT research scientist who is now the chief technology officer at SanaHeal. “It was such an exciting idea, and the rest was just making it happen.”

The team, comprising researchers affiliated with the lab of Xuanhe Zhao, professor of mechanical engineering and of civil and environmental engineering, has introduced a 3D-printable bioadhesive pacing lead that can directly interface with cardiac tissue, supporting minimally invasive adhesive implantation and providing a detachment solution that allows for gentle removal. Yuk and Zhao are the corresponding authors of the study; former MIT researcher Jue Deng is the paper’s first author.

“This work introduces the first on-demand detachable bioadhesive version of temporary cardiac pacing lead that offers atraumatic application and removal of the device with enhanced safety while offering improved bioelectronic performance,” says Zhao.

The development of the bioadhesive pacing lead is a combination of technologies that the team has developed over the last several years in the field of bioadhesive, bioelectronics, and 3D printing. SanaHeal, a company born from the team’s ongoing work, is commercializing bioadhesive technologies for various clinical applications.

“We hope that our ongoing effort on commercialization of our bioadhesive technology might help faster clinical translation of our bioadhesive pacing lead as well,” says Yuk.

The tenured engineers of 2024

The tenured engineers of 2024

In 2024, MIT granted tenure to 11 faculty members across the School of Engineering. This year’s tenured engineers hold appointments in the departments of Aeronautics and Astronautics, Chemical Engineering, Civil and Environmental Engineering, Electrical Engineering and Computer Science (EECS, which reports jointly to the School of Engineering and MIT Schwarzman College of Computing), Mechanical Engineering, and Nuclear Science and Engineering.

“My heartfelt congratulations to the 11 engineering faculty members on receiving tenure. These faculty have already made a lasting impact in the School of Engineering through both advances in their field and their dedication as educators and mentors,” says Anantha Chandrakasan, chief innovation and strategy officer, dean of engineering, and the Vannevar Bush Professor of Electrical Engineering and Computer Science.

This year’s newly tenured engineering faculty include:

Adam Belay, associate professor of computer science and principal investigator at MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL), works on operating systems, runtime systems, and distributed systems. He is particularly interested in developing practical methods for microsecond-scale computing and cloud resource management, with many applications relating to performance and computing efficiency within large data centers.

Irmgard Bischofberger, Class of 1942 Career Development Professor and associate professor of mechanical engineering, is an expert in the mechanisms of pattern formation and instabilities in complex fluids. Her research reveals new insights into classical understanding of instabilities and has wide relevance to physical systems and industrial processes. Further, she is dedicated to science communication and generates exquisite visualizations of complex fluidic phenomena from her research.

Matteo Bucci serves as the Esther and Harold E. Edgerton Associate Professor of nuclear science and engineering. His research group studies two-phase heat transfer mechanisms in nuclear reactors and space systems, develops high-resolution, nonintrusive diagnostics and surface engineering techniques to enhance two-phase heat transfer, and creates machine-learning tools to accelerate data analysis and conduct autonomous heat transfer experiments.

Luca Carlone, the Boeing Career Development Professor in Aeronautics and Astronautics, is head of the Sensing, Perception, Autonomy, and Robot Kinetics Laboratory and principal investigator at the Laboratory for Information and Decision Systems. His research focuses on the cutting edge of robotics and autonomous systems research, with a particular interest in designing certifiable perception algorithms for high-integrity autonomous systems and developing algorithms and systems for real-time 3D scene understanding on mobile robotics platforms operating in the real world.

Manya Ghobadi, associate professor of computer science and principal investigator at CSAIL, builds efficient network infrastructures that optimize resource use, energy consumption, and availability of large-scale systems. She is a leading expert in networks with reconfigurable physical layers, and many of the ideas she has helped develop are part of real-world systems.

Zachary (Zach) Hartwig serves as the Robert N. Noyce Career Development Professor in the Department of Nuclear Science and Engineering, with a co-appointment at MIT’s Plasma Science and Fusion Center. His current research focuses on the development of high-field superconducting magnet technologies for fusion energy and accelerated irradiation methods for fusion materials using ion beams. He is a co-founder of Commonwealth Fusion Systems, a private company commercializing fusion energy.

Admir Masic, associate professor of civil and environmental engineering, focuses on bridging the gap between ancient wisdom and modern material technologies. He applies his expertise in the fields of in situ and operando spectroscopic techniques to develop sustainable materials for construction, energy, and the environment.

Stefanie Mueller is the TIBCO Career Development Professor in the Department of EECS. Mueller has a joint appointment in the Department of Mechanical Engineering and is a principal investigator at CSAIL. She develops novel hardware and software systems that give objects new capabilities. Among other applications, her lab creates health sensing devices and electronic sensing devices for curved surfaces; embedded sensors; fabrication techniques that enable objects to be trackable via invisible marker; and objects with reprogrammable and interactive appearances.

Koroush Shirvan serves as the Atlantic Richfield Career Development Professor in Energy Studies in the Department of Nuclear Science and Engineering. He specializes in the development and assessment of advanced nuclear reactor technology. He is currently focused on accelerating innovations in nuclear fuels, reactor design, and small modular reactors to improve the sustainability of current and next-generation power plants. His approach combines multiple scales, physics and disciplines to realize innovative solutions in the highly regulated nuclear energy sector.

Julian Shun, associate professor of computer science and principal investigator at CSAIL, focuses on the theory and practice of parallel and high-performance computing. He is interested in designing algorithms that are efficient in both theory and practice, as well as high-level frameworks that make it easier for programmers to write efficient parallel code. His research has focused on designing solutions for graphs, spatial data, and dynamic problems.

Zachary P. Smith, Robert N. Noyce Career Development Professor and associate professor of chemical engineering, focuses on the molecular-level design, synthesis, and characterization of polymers and inorganic materials for applications in membrane-based separations, which is a promising aid for the energy industry and the environment, from dissolving olefins found in plastics or rubber, to capturing smokestack carbon dioxide emissions. He is a co-founder and chief scientist of Osmoses, a startup aiming to commercialize membrane technology for industrial gas separations.

Rob Clark, President and CTO of Seekr – Interview Series

Rob Clark is the President and Chief Technology Office (CTO) of Seekr. Rob has over 20 years of experience in software engineering, product management, operations, and the development of leading-edge artificial intelligence and web-scale technologies. Before joining Seekr, he led several artificial intelligence and search solutions…

AI Auditing: Ensuring Performance and Accuracy in Generative Models

In recent years, the world has witnessed the unprecedented rise of Artificial Intelligence (AI), which has transformed numerous sectors and reshaped our everyday lives. Among the most transformative advancements are generative models, AI systems capable of creating text, images, music, and more with surprising creativity and…

“Rollerama” roller rink opens in Kendall Square

“Rollerama” roller rink opens in Kendall Square

The former U.S. Department of Transportation (DOT) Volpe Center site — now named “Kendall Common” in anticipation of its transformation into a vibrant mixed-use development — is now activated and open to all this summer. “Rollerama at Kendall Common” offers free roller-skating and roller skate rentals, community programming, and family-friendly events through September.

“We are extremely excited to bring Kendall Common to life in a way that is inviting and authentically Cambridge, while channeling MIT’s spirit of innovation throughout the project,” says Patrick Rowe, senior vice president, MIT Investment Management Co. “This parcel of land — right in the heart of Kendall Square — has been closed off to local residents and visitors for far too long, and we look forward to opening it up and making it accessible for all to utilize and enjoy.”

Located at the corner of Broadway and Third Street, Rollerama offers specialty themed skating nights and live entertainment, as well as food and beverage from local restaurants for purchase. Optional skate rental donations will be directed to local nonprofits. A highlight of the space is a new 7,000 square foot mural by Boston-based artist Massiel Grullón featuring retro-inspired shapes.

The first of two opening weekends took place June 28-30; the next one will be July 5-7 from 2-8 p.m. on Fridays, and 11 a.m. to 8 p.m. on Saturdays and Sundays. From July 10 through Sept. 29, Rollerama will be open Wednesdays, Thursdays, and Fridays from 2-8 p.m., and on Saturdays and Sundays from 11 a.m. to 8 p.m.

“We’re delighted to see this underutilized space activated with vibrant and playful programming,” says Jess Smith, director of MIT Open Space Programming. “Rollerama will add to the energy of Kendall Square and provide yet another compelling reason for employees, residents, students, and visitors to mix and mingle here. With food and drink available from Cambridge partners and voluntary donations going to Cambridge nonprofits, these activities in Kendall Common will contribute significantly to the sense of community in Kendall.”

The activation of Kendall Common will complement other new additions MIT has recently brought to the Kendall Square neighborhood, including Ripple Café, Row 34, Life Alive Café, Locke Bar, and Flat Top Johnny’s, along with the MIT Museum and MIT Press Book Store.

MIT assumed ownership of 10 acres of the former U.S. DOT Volpe Center site in Kendall Square earlier this year, and will commence infrastructure and site preparation for the redevelopment this fall. Over the coming years, MIT aims to transform Kendall Common into a vibrant, mixed-use development that will strengthen connections in the Cambridge community through new open green spaces, housing, retail offerings, restaurants, a community center, and science and innovation facilities.

Kendall Common will eventually include four residential buildings, four commercial buildings, four parks and a community center. Designed to be an inclusive and equitable urban environment with a focus on sustainability, the development is intended to nurture and inspire the local community.

For more information visit the Kendall Common website, Instagram page, and Facebook page.

A prosthesis driven by the nervous system helps people with amputation walk naturally

State-of-the-art prosthetic limbs can help people with amputations achieve a natural walking gait, but they don’t give the user full neural control over the limb. Instead, they rely on robotic sensors and controllers that move the limb using predefined gait algorithms.

Using a new type of surgical intervention and neuroprosthetic interface, MIT researchers, in collaboration with colleagues from Brigham and Women’s Hospital, have shown that a natural walking gait is achievable using a prosthetic leg fully driven by the body’s own nervous system. The surgical amputation procedure reconnects muscles in the residual limb, which allows patients to receive “proprioceptive” feedback about where their prosthetic limb is in space.

In a study of seven patients who had this surgery, the MIT team found that they were able to walk faster, avoid obstacles, and climb stairs much more naturally than people with a traditional amputation.

“This is the first prosthetic study in history that shows a leg prosthesis under full neural modulation, where a biomimetic gait emerges. No one has been able to show this level of brain control that produces a natural gait, where the human’s nervous system is controlling the movement, not a robotic control algorithm,” says Hugh Herr, a professor of media arts and sciences, co-director of the K. Lisa Yang Center for Bionics at MIT, an associate member of MIT’s McGovern Institute for Brain Research, and the senior author of the new study.

Patients also experienced less pain and less muscle atrophy following this surgery, which is known as the agonist-antagonist myoneural interface (AMI). So far, about 60 patients around the world have received this type of surgery, which can also be done for people with arm amputations.

Hyungeun Song, a postdoc in MIT’s Media Lab, is the lead author of the paper, which appears today in Nature Medicine.

Sensory feedback

Most limb movement is controlled by pairs of muscles that take turns stretching and contracting. During a traditional below-the-knee amputation, the interactions of these paired muscles are disrupted. This makes it very difficult for the nervous system to sense the position of a muscle and how fast it’s contracting — sensory information that is critical for the brain to decide how to move the limb.

People with this kind of amputation may have trouble controlling their prosthetic limb because they can’t accurately sense where the limb is in space. Instead, they rely on robotic controllers built into the prosthetic limb. These limbs also include sensors that can detect and adjust to slopes and obstacles.

To try to help people achieve a natural gait under full nervous system control, Herr and his colleagues began developing the AMI surgery several years ago. Instead of severing natural agonist-antagonist muscle interactions, they connect the two ends of the muscles so that they still dynamically communicate with each other within the residual limb. This surgery can be done during a primary amputation, or the muscles can be reconnected after the initial amputation as part of a revision procedure.

“With the AMI amputation procedure, to the greatest extent possible, we attempt to connect native agonists to native antagonists in a physiological way so that after amputation, a person can move their full phantom limb with physiologic levels of proprioception and range of movement,” Herr says.

In a 2021 study, Herr’s lab found that patients who had this surgery were able to more precisely control the muscles of their amputated limb, and that those muscles produced electrical signals similar to those from their intact limb.

After those encouraging results, the researchers set out to explore whether those electrical signals could generate commands for a prosthetic limb and at the same time give the user feedback about the limb’s position in space. The person wearing the prosthetic limb could then use that proprioceptive feedback to volitionally adjust their gait as needed.

In the new Nature Medicine study, the MIT team found this sensory feedback did indeed translate into a smooth, near-natural ability to walk and navigate obstacles.

“Because of the AMI neuroprosthetic interface, we were able to boost that neural signaling, preserving as much as we could. This was able to restore a person’s neural capability to continuously and directly control the full gait, across different walking speeds, stairs, slopes, even going over obstacles,” Song says.

A natural gait

For this study, the researchers compared seven people who had the AMI surgery with seven who had traditional below-the-knee amputations. All of the subjects used the same type of bionic limb: a prosthesis with a powered ankle as well as electrodes that can sense electromyography (EMG) signals from the tibialis anterior the gastrocnemius muscles. These signals are fed into a robotic controller that helps the prosthesis calculate how much to bend the ankle, how much torque to apply, or how much power to deliver.

The researchers tested the subjects in several different situations: level-ground walking across a 10-meter pathway, walking up a slope, walking down a ramp, walking up and down stairs, and walking on a level surface while avoiding obstacles.

In all of these tasks, the people with the AMI neuroprosthetic interface were able to walk faster — at about the same rate as people without amputations — and navigate around obstacles more easily. They also showed more natural movements, such as pointing the toes of the prosthesis upward while going up stairs or stepping over an obstacle, and they were better able to coordinate the movements of their prosthetic limb and their intact limb. They were also able to push off the ground with the same amount of force as someone without an amputation.

“With the AMI cohort, we saw natural biomimetic behaviors emerge,” Herr says. “The cohort that didn’t have the AMI, they were able to walk, but the prosthetic movements weren’t natural, and their movements were generally slower.”

These natural behaviors emerged even though the amount of sensory feedback provided by the AMI was less than 20 percent of what would normally be received in people without an amputation.

“One of the main findings here is that a small increase in neural feedback from your amputated limb can restore significant bionic neural controllability, to a point where you allow people to directly neurally control the speed of walking, adapt to different terrain, and avoid obstacles,” Song says.

“This work represents yet another step in us demonstrating what is possible in terms of restoring function in patients who suffer from severe limb injury. It is through collaborative efforts such as this that we are able to make transformational progress in patient care,” says Matthew Carty, a surgeon at Brigham and Women’s Hospital and associate professor at Harvard Medical School, who is also an author of the paper.

Enabling neural control by the person using the limb is a step toward Herr’s lab’s goal of “rebuilding human bodies,” rather than having people rely on ever more sophisticated robotic controllers and sensors — tools that are powerful but do not feel like part of the user’s body.

“The problem with that long-term approach is that the user would never feel embodied with their prosthesis. They would never view the prosthesis as part of their body, part of self,” Herr says. “The approach we’re taking is trying to comprehensively connect the brain of the human to the electromechanics.”

The research was funded by the MIT K. Lisa Yang Center for Bionics and the Eunice Kennedy Shriver National Institute of Child Health and Human Development.