Consuming content efficiently has become a critical skill. YouTube, being the world’s largest video-sharing platform, offers an endless stream of valuable content across various domains. However, sifting through lengthy videos to extract key insights can be time-consuming and cumbersome. This is where AI-powered YouTube summarizer tools…
A faster, better way to prevent an AI chatbot from giving toxic responses
A user could ask ChatGPT to write a computer program or summarize an article, and the AI chatbot would likely be able to generate useful code or write a cogent synopsis. However, someone could also ask for instructions to build a bomb, and the chatbot might be able to provide those, too.
To prevent this and other safety issues, companies that build large language models typically safeguard them using a process called red-teaming. Teams of human testers write prompts aimed at triggering unsafe or toxic text from the model being tested. These prompts are used to teach the chatbot to avoid such responses.
But this only works effectively if engineers know which toxic prompts to use. If human testers miss some prompts, which is likely given the number of possibilities, a chatbot regarded as safe might still be capable of generating unsafe answers.
Researchers from Improbable AI Lab at MIT and the MIT-IBM Watson AI Lab used machine learning to improve red-teaming. They developed a technique to train a red-team large language model to automatically generate diverse prompts that trigger a wider range of undesirable responses from the chatbot being tested.
They do this by teaching the red-team model to be curious when it writes prompts, and to focus on novel prompts that evoke toxic responses from the target model.
The technique outperformed human testers and other machine-learning approaches by generating more distinct prompts that elicited increasingly toxic responses. Not only does their method significantly improve the coverage of inputs being tested compared to other automated methods, but it can also draw out toxic responses from a chatbot that had safeguards built into it by human experts.
“Right now, every large language model has to undergo a very lengthy period of red-teaming to ensure its safety. That is not going to be sustainable if we want to update these models in rapidly changing environments. Our method provides a faster and more effective way to do this quality assurance,” says Zhang-Wei Hong, an electrical engineering and computer science (EECS) graduate student in the Improbable AI lab and lead author of a paper on this red-teaming approach.
Hong’s co-authors include EECS graduate students Idan Shenfield, Tsun-Hsuan Wang, and Yung-Sung Chuang; Aldo Pareja and Akash Srivastava, research scientists at the MIT-IBM Watson AI Lab; James Glass, senior research scientist and head of the Spoken Language Systems Group in the Computer Science and Artificial Intelligence Laboratory (CSAIL); and senior author Pulkit Agrawal, director of Improbable AI Lab and an assistant professor in CSAIL. The research will be presented at the International Conference on Learning Representations.
Automated red-teaming
Large language models, like those that power AI chatbots, are often trained by showing them enormous amounts of text from billions of public websites. So, not only can they learn to generate toxic words or describe illegal activities, the models could also leak personal information they may have picked up.
The tedious and costly nature of human red-teaming, which is often ineffective at generating a wide enough variety of prompts to fully safeguard a model, has encouraged researchers to automate the process using machine learning.
Such techniques often train a red-team model using reinforcement learning. This trial-and-error process rewards the red-team model for generating prompts that trigger toxic responses from the chatbot being tested.
But due to the way reinforcement learning works, the red-team model will often keep generating a few similar prompts that are highly toxic to maximize its reward.
For their reinforcement learning approach, the MIT researchers utilized a technique called curiosity-driven exploration. The red-team model is incentivized to be curious about the consequences of each prompt it generates, so it will try prompts with different words, sentence patterns, or meanings.
“If the red-team model has already seen a specific prompt, then reproducing it will not generate any curiosity in the red-team model, so it will be pushed to create new prompts,” Hong says.
During its training process, the red-team model generates a prompt and interacts with the chatbot. The chatbot responds, and a safety classifier rates the toxicity of its response, rewarding the red-team model based on that rating.
Rewarding curiosity
The red-team model’s objective is to maximize its reward by eliciting an even more toxic response with a novel prompt. The researchers enable curiosity in the red-team model by modifying the reward signal in the reinforcement learning set up.
First, in addition to maximizing toxicity, they include an entropy bonus that encourages the red-team model to be more random as it explores different prompts. Second, to make the agent curious they include two novelty rewards. One rewards the model based on the similarity of words in its prompts, and the other rewards the model based on semantic similarity. (Less similarity yields a higher reward.)
To prevent the red-team model from generating random, nonsensical text, which can trick the classifier into awarding a high toxicity score, the researchers also added a naturalistic language bonus to the training objective.
With these additions in place, the researchers compared the toxicity and diversity of responses their red-team model generated with other automated techniques. Their model outperformed the baselines on both metrics.
They also used their red-team model to test a chatbot that had been fine-tuned with human feedback so it would not give toxic replies. Their curiosity-driven approach was able to quickly produce 196 prompts that elicited toxic responses from this “safe” chatbot.
“We are seeing a surge of models, which is only expected to rise. Imagine thousands of models or even more and companies/labs pushing model updates frequently. These models are going to be an integral part of our lives and it’s important that they are verified before released for public consumption. Manual verification of models is simply not scalable, and our work is an attempt to reduce the human effort to ensure a safer and trustworthy AI future,” says Agrawal.
In the future, the researchers want to enable the red-team model to generate prompts about a wider variety of topics. They also want to explore the use of a large language model as the toxicity classifier. In this way, a user could train the toxicity classifier using a company policy document, for instance, so a red-team model could test a chatbot for company policy violations.
“If you are releasing a new AI model and are concerned about whether it will behave as expected, consider using curiosity-driven red-teaming,” says Agrawal.
This research is funded, in part, by Hyundai Motor Company, Quanta Computer Inc., the MIT-IBM Watson AI Lab, an Amazon Web Services MLRA research grant, the U.S. Army Research Office, the U.S. Defense Advanced Research Projects Agency Machine Common Sense Program, the U.S. Office of Naval Research, the U.S. Air Force Research Laboratory, and the U.S. Air Force Artificial Intelligence Accelerator.
How artificial intelligence is revolutionizing cyber security – CyberTalk
By Shira Landau, Editor-in-Chief, CyberTalk.org.
In recent years, artificial intelligence (AI) has become one of the most sure-fire and strategic tools available for cyber security professionals. Due to the increasing sophistication of cyber attacks, cyber security experts have broadly turned to AI in order to enhance abilities to detect and prevent cyber threats.
As it stands, nearly 50% of enterprises are already using a combination of artificial intelligence and machine learning tools to improve cyber security outcomes, and 92% of organizations plan to adopt these types of tools in the future.
Powerful AI technology is particularly useful for identifying and mitigating security threats that are difficult or impossible to detect manually, such as zero-day exploits, polymorphic malware, and advanced persistent threats. AI-based tools can also help streamline tasks, lower costs, augment under-resourced operations and enable security professionals to work ‘smarter.’
Are you ready to take your organization’s cyber security to the next level? With AI, you can stay ahead of the curve and protect your organization from the most advanced of cyber threats. In this article, explore the incredible ways in which AI is enhancing and revolutionizing cyber security and the digital world.
Key information
- A spike in cyber attacks has helped fuel market growth for AI-based cyber security products.
- The global market for AI-based cyber security products is estimated to reach $133.8 billion by 2030.
- AI-based tools enable cyber security professionals to work smarter and more efficiently than is otherwise possible.
How AI is revolutionizing cyber security
1. Threat detection. One of the most significant challenges that cyber security professionals face is the sheer volume of data that they need to sift through. Given the number of internet-connected devices (IoT growth is projected to reach 3.22 billion in North America alone in 2023), there is a seemingly insatiable appetite for data processing.
Artificial intelligence technology is extremely helpful when it comes to efficiently and accurately analyzing large volumes of data, rendering AI an essential tool for cyber security professionals. Algorithms can quickly analyze patterns in data to identify threats and to detect anomalous behavior.
2. Automation. AI is also being deployed in order to automate and streamline aspects of cyber security. In turn, this enables cyber security professionals to focus on investigating and mitigating complex threats, while AI takes care of tedious or monotonous basic tasks.
3. Machine learning. Another advantage of AI-powered cyber security systems consists of its ability to learn from past attacks and to improve on existing threat detection capabilities.
By looking at data from past attacks, machine learning algorithms can identify patterns, and then actually develop new and sophisticated detection methods. Over time, this development makes breaching systems tougher for cyber criminals.
4. Insider threats. Artificial intelligence is particularly useful in cyber security when it comes to detecting and responding to insider threats. These threats are tricky to detect, as the individuals involved always have legitimate access to a given network.
Nonetheless, AI-powered systems can analyze user behavior, and thereby identify patterns that indicate an insider threat. Such patterns can then be flagged for further investigation.
5. Endpoint security. AI is also being used to enhance endpoint security. Traditional endpoint security solutions rely on signature-based detection, which involves identifying known threats and blocking them. But this approach is losing its effectiveness.
AI-powered endpoint security solutions leverage machine learning algorithms to identify anomalous behavior and to detect previously unknown threats. This approach is more effective than what traditional endpoint security solutions can offer, as it can identify threats that would otherwise remain unnoticed.
6. Finally, AI is being used to improve threat intelligence. By analyzing large volumes of data from disparate sources, AI-powered threat intelligence solutions can zero in on potential threats and offer early warnings around new types of attacks. This information can then be used to develop optimally effective cyber security strategies and to advance the overall security posture of an organization.
In conclusion
AI is revolutionizing the field of cyber security by providing cyber security professionals with the tools that they need to detect, prevent and respond to cyber threats.
Are you drowning in data? Struggling to keep up amidst the ever-evolving threat landscape? Get ready for whatever comes your way! Explore AI-based cyber security tools that make it easier (and more efficient) than ever to protect your systems. Click here to learn more and to start applying AI’s game-changing capabilities within your business.
Want to stay up-to-date with the latest and greatest in cyber security? Check out the CyberTalk.org newsletter! Sign up today to receive top-notch news articles, best practices and expert analyses; delivered straight to your inbox.
Stability AI unveils 12B parameter Stable LM 2 model and updated 1.6B variant
Stability AI has introduced the latest additions to its Stable LM 2 language model series: a 12 billion parameter base model and an instruction-tuned variant. These models were trained on an impressive two trillion tokens across seven languages: English, Spanish, German, Italian, French, Portuguese, and Dutch….
ChatGPT & the Enterprise: Balancing Caution and Innovation in the Age of AI
OpenAI’s groundbreaking AI tool ChatGPT was officially launched on November 30th, 2022. However, it wasn’t until the early months of 2023 that its impact truly began to ripple through the global consciousness. This transition from a novel technological release to a sensation that captivated the world…
Enhancing AI Transparency and Trust with Composite AI
The adoption of Artificial Intelligence (AI) has increased rapidly across domains such as healthcare, finance, and legal systems. However, this surge in AI usage has raised concerns about transparency and accountability. Several times black-box AI models have produced unintended consequences, including biased decisions and lack of…
HyperCLOVA X: Naver’s leap in Asian language AI
Naver unveils HyperCLOVA X, a large language model excelling in Asian languages, aiming for global AI advancements and sovereign tech development….
Extracting hydrogen from rocks
It’s commonly thought that the most abundant element in the universe, hydrogen, exists mainly alongside other elements — with oxygen in water, for example, and with carbon in methane. But naturally occurring underground pockets of pure hydrogen are punching holes in that notion — and generating attention as a potentially unlimited source of carbon-free power.
One interested party is the U.S. Department of Energy, which last month awarded $20 million in research grants to 18 teams from laboratories, universities, and private companies to develop technologies that can lead to cheap, clean fuel from the subsurface.
Geologic hydrogen, as it’s known, is produced when water reacts with iron-rich rocks, causing the iron to oxidize. One of the grant recipients, MIT Assistant Professor Iwnetim Abate’s research group, will use its $1.3 million grant to determine the ideal conditions for producing hydrogen underground — considering factors such as catalysts to initiate the chemical reaction, temperature, pressure, and pH levels. The goal is to improve efficiency for large-scale production, meeting global energy needs at a competitive cost.
The U.S. Geological Survey estimates there are potentially billions of tons of geologic hydrogen buried in the Earth’s crust. Accumulations have been discovered worldwide, and a slew of startups are searching for extractable deposits. Abate is looking to jump-start the natural hydrogen production process, implementing “proactive” approaches that involve stimulating production and harvesting the gas.
“We aim to optimize the reaction parameters to make the reaction faster and produce hydrogen in an economically feasible manner,” says Abate, the Chipman Development Professor in the Department of Materials Science and Engineering (DMSE). Abate’s research centers on designing materials and technologies for the renewable energy transition, including next-generation batteries and novel chemical methods for energy storage.
Sparking innovation
Interest in geologic hydrogen is growing at a time when governments worldwide are seeking carbon-free energy alternatives to oil and gas. In December, French President Emmanuel Macron said his government would provide funding to explore natural hydrogen. And in February, government and private sector witnesses briefed U.S. lawmakers on opportunities to extract hydrogen from the ground.
Today commercial hydrogen is manufactured at $2 a kilogram, mostly for fertilizer and chemical and steel production, but most methods involve burning fossil fuels, which release Earth-heating carbon. “Green hydrogen,” produced with renewable energy, is promising, but at $7 per kilogram, it’s expensive.
“If you get hydrogen at a dollar a kilo, it’s competitive with natural gas on an energy-price basis,” says Douglas Wicks, a program director at Advanced Research Projects Agency – Energy (ARPA-E), the Department of Energy organization leading the geologic hydrogen grant program.
Recipients of the ARPA-E grants include Colorado School of Mines, Texas Tech University, and Los Alamos National Laboratory, plus private companies including Koloma, a hydrogen production startup that has received funding from Amazon and Bill Gates. The projects themselves are diverse, ranging from applying industrial oil and gas methods for hydrogen production and extraction to developing models to understand hydrogen formation in rocks. The purpose: to address questions in what Wicks calls a “total white space.”
“In geologic hydrogen, we don’t know how we can accelerate the production of it, because it’s a chemical reaction, nor do we really understand how to engineer the subsurface so that we can safely extract it,” Wicks says. “We’re trying to bring in the best skills of each of the different groups to work on this under the idea that the ensemble should be able to give us good answers in a fairly rapid timeframe.”
Geochemist Viacheslav Zgonnik, one of the foremost experts in the natural hydrogen field, agrees that the list of unknowns is long, as is the road to the first commercial projects. But he says efforts to stimulate hydrogen production — to harness the natural reaction between water and rock — present “tremendous potential.”
“The idea is to find ways we can accelerate that reaction and control it so we can produce hydrogen on demand in specific places,” says Zgonnik, CEO and founder of Natural Hydrogen Energy, a Denver-based startup that has mineral leases for exploratory drilling in the United States. “If we can achieve that goal, it means that we can potentially replace fossil fuels with stimulated hydrogen.”
“A full-circle moment”
For Abate, the connection to the project is personal. As a child in his hometown in Ethiopia, power outages were a usual occurrence — the lights would be out three, maybe four days a week. Flickering candles or pollutant-emitting kerosene lamps were often the only source of light for doing homework at night.
“And for the household, we had to use wood and charcoal for chores such as cooking,” says Abate. “That was my story all the way until the end of high school and before I came to the U.S. for college.”
In 1987, well-diggers drilling for water in Mali in Western Africa uncovered a natural hydrogen deposit, causing an explosion. Decades later, Malian entrepreneur Aliou Diallo and his Canadian oil and gas company tapped the well and used an engine to burn hydrogen and power electricity in the nearby village.
Ditching oil and gas, Diallo launched Hydroma, the world’s first hydrogen exploration enterprise. The company is drilling wells near the original site that have yielded high concentrations of the gas.
“So, what used to be known as an energy-poor continent now is generating hope for the future of the world,” Abate says. “Learning about that was a full-circle moment for me. Of course, the problem is global; the solution is global. But then the connection with my personal journey, plus the solution coming from my home continent, makes me personally connected to the problem and to the solution.”
Experiments that scale
Abate and researchers in his lab are formulating a recipe for a fluid that will induce the chemical reaction that triggers hydrogen production in rocks. The main ingredient is water, and the team is testing “simple” materials for catalysts that will speed up the reaction and in turn increase the amount of hydrogen produced, says postdoc Yifan Gao.
“Some catalysts are very costly and hard to produce, requiring complex production or preparation,” Gao says. “A catalyst that’s inexpensive and abundant will allow us to enhance the production rate — that way, we produce it at an economically feasible rate, but also with an economically feasible yield.”
The iron-rich rocks in which the chemical reaction happens can be found across the United States and the world. To optimize the reaction across a diversity of geological compositions and environments, Abate and Gao are developing what they call a high-throughput system, consisting of artificial intelligence software and robotics, to test different catalyst mixtures and simulate what would happen when applied to rocks from various regions, with different external conditions like temperature and pressure.
“And from that we measure how much hydrogen we are producing for each possible combination,” Abate says. “Then the AI will learn from the experiments and suggest to us, ‘Based on what I’ve learned and based on the literature, I suggest you test this composition of catalyst material for this rock.’”
The team is writing a paper on its project and aims to publish its findings in the coming months.
The next milestones for the project, after developing the catalyst recipe, is designing a reactor that will serve two purposes. First, fitted with technologies such as Raman spectroscopy, it will allow researchers to identify and optimize the chemical conditions that lead to improved rates and yield of hydrogen production. The lab-scale device will also inform the design of a real-world reactor that can accelerate hydrogen production in the field.
“That would be a plant-scale reactor that would be implanted into the subsurface,” Abate says.
The cross-disciplinary project is also tapping the expertise of Yang Shao-Horn, of MIT’s Department of Mechanical Engineering and DMSE, for computational analysis of the catalyst, and Esteban Gazel, a Cornell University scientist who will lend his expertise in geology and geochemistry. He’ll focus on understanding the iron-rich ultramafic rock formations across the United States and the globe and how they react with water.
For Wicks at ARPA-E, the questions Abate and the other grant recipients are asking are just the first, critical steps in uncharted energy territory.
“If we can understand how to stimulate these rocks into generating hydrogen, safely getting it up, it really unleashes the potential energy source,” he says. Then the emerging industry will look to oil and gas for the drilling, piping, and gas extraction know-how. “As I like to say, this is enabling technology that we hope to, in a very short term, enable us to say, ‘Is there really something there?’”
When an antibiotic fails: MIT scientists are using AI to target “sleeper” bacteria
Since the 1970s, modern antibiotic discovery has been experiencing a lull. Now the World Health Organization has declared the antimicrobial resistance crisis as one of the top 10 global public health threats.
When an infection is treated repeatedly, clinicians run the risk of bacteria becoming resistant to the antibiotics. But why would an infection return after proper antibiotic treatment? One well-documented possibility is that the bacteria are becoming metabolically inert, escaping detection of traditional antibiotics that only respond to metabolic activity. When the danger has passed, the bacteria return to life and the infection reappears.
“Resistance is happening more over time, and recurring infections are due to this dormancy,” says Jackie Valeri, a former MIT-Takeda Fellow (centered within the MIT Abdul Latif Jameel Clinic for Machine Learning in Health) who recently earned her PhD in biological engineering from the Collins Lab. Valeri is the first author of a new paper published in this month’s print issue of Cell Chemical Biology that demonstrates how machine learning could help screen compounds that are lethal to dormant bacteria.
Tales of bacterial “sleeper-like” resilience are hardly news to the scientific community — ancient bacterial strains dating back to 100 million years ago have been discovered in recent years alive in an energy-saving state on the seafloor of the Pacific Ocean.
MIT Jameel Clinic’s Life Sciences faculty lead James J. Collins, a Termeer Professor of Medical Engineering and Science in MIT’s Institute for Medical Engineering and Science and Department of Biological Engineering, recently made headlines for using AI to discover a new class of antibiotics, which is part of the group’s larger mission to use AI to dramatically expand the existing antibiotics available.
According to a paper published by The Lancet, in 2019, 1.27 million deaths could have been prevented had the infections been susceptible to drugs, and one of many challenges researchers are up against is finding antibiotics that are able to target metabolically dormant bacteria.
In this case, researchers in the Collins Lab employed AI to speed up the process of finding antibiotic properties in known drug compounds. With millions of molecules, the process can take years, but researchers were able to identify a compound called semapimod over a weekend, thanks to AI’s ability to perform high-throughput screening.
An anti-inflammatory drug typically used for Crohn’s disease, researchers discovered that semapimod was also effective against stationary-phase Escherichia coli and Acinetobacter baumannii.
Another revelation was semapimod’s ability to disrupt the membranes of so-called “Gram-negative” bacteria, which are known for their high intrinsic resistance to antibiotics due to their thicker, less-penetrable outer membrane.
Examples of Gram-negative bacteria include E. coli, A. baumannii, Salmonella, and Pseudomonis, all of which are challenging to find new antibiotics for.
“One of the ways we figured out the mechanism of sema [sic] was that its structure was really big, and it reminded us of other things that target the outer membrane,” Valeri explains. “When you start working with a lot of small molecules … to our eyes, it’s a pretty unique structure.”
By disrupting a component of the outer membrane, semapimod sensitizes Gram-negative bacteria to drugs that are typically only active against Gram-positive bacteria.
Valeri recalls a quote from a 2013 paper published in Trends Biotechnology: “For Gram-positive infections, we need better drugs, but for Gram-negative infections we need any drugs.”
Unlocking AI: 8 key steps from POC to success
Join Sol Rashidi at the San Jose Summit for insights on AI innovation and leadership, and learn from her journey from data analyst to tech visionary….