Using AI to Co-develop New Product Formulations and the Future of Flavor Innovation

Good morning, uh welcome to this months version of the webinar series on applied analytics success stories. Uh. My name is rajesh thiagi and im the organizer for this series. These webinars are an initiative of the informs practice session and on this particular occasion we are co-hosting it with the informs women in orms forum and dr olgar parikhaki is the president of that forum. Now before we get on to the speaker on todays topic uh. Let me give a very

Brief update or or an overview of, the inform section on practice. I hope you guys can see my screen here. So the objective of the practice session is to advance the practice of analytics operations, research and the management sciences and to support that mission. This section conducts a number of activities, so in the on the top, you have the three uh annual competitions which are shown partially in blue font, and i guess most of them most of you or many of you would

Be familiar with some of those competitions, we also have some monthly events, so we have the applied analytics success stories uh, the next one is going to be. In december 17th, we have a relatively new series called sustaining outstanding analytics organizations, and the focus here is on the organization itself. The way the analytics organization is set up, the kind of people they have, the skills, the projects uh. The next one will take place on

November 18th and i believe facebook will be presented there. And thirdly, we have the monthly virtual networking happy hour, which is held on a friday last friday of each month, uh because of the annual meeting its going to be held on october 26 uh this particular month. At a special time of 8, 45 um again, if you’re interested in participating in some of these competitions or learning about these or

want to participate in some of the monthly webinars go to the web page for the practice section learn a little bit more about that and and definitely remember to register for these webinars uh and before we go to robin let me just hand it over to olga all guys going to talk a little bit about the women in orms forum olga thank you so much first of all i would like to thank you for giving me the opportunity to um and go warms the.

opportunity to work on this initiative together so as i just mentioned im the current president of the women in operations research and the management sciences forum would you mind registering yes im sorry okay great and uh for those of you who may not be familiar with the forum it was established back in 1995 and the whole idea was conceived over a um over a brown bag lunch so the purpose of this forum is to promote interest in the issues facing women and the relationship to the profession of operations research and the management sciences.

And, the forum organizes a number of events uh around the two main informs conferences the annual meeting um in the fall as well as the business analytics conference in the spring. So in a typical informs annual meeting we. Will have the worm sponsored sessions that will be typically a presentation session and a panel discussion. We will have the worms luncheon that will give the opportunity to worms members to socialize in a relaxed environment and during the worms luncheons. We have a ceremony of the different types of awards that worms is presenting.

so these are the list of the different types of awards that uh worms is presenting during the luncheon the awards for the advancement of women in operations research and management sciences for those of you who are not aware of this particular award it actually recognizes individuals who have contributed significantly to the advancement of women in the field of operations research and management sciences and depending on the name on the nominations we typically offer one award to an individual from industry and then one award to an individual from.

academia then we have the doctoral collocum award that supports a phd student to attend the doctoral colloquium at the annual meeting and the family care award that partially covers expenses related to care and travel for family members whose main caregivers are attending the annual meeting we have a business meeting where we discuss the step the status of our forum followed typically by networking reception or our networking interception has become so popular over the past few years that i remember back in 2019 we had a very long line of people waiting to enter the.

venue and then we have a mentorship program called coffee with a mentor that gives the opportunity to our worms members to connect with a member with a mentor so that they can get information advice and guidance related to their career both in industry as well as in academia we also have a subset of similar type of events uh uh happening at the business.

analytics conference uh since epidemic had started we have also started a few webinars and you can see some of our webinars in our worms webpage you can listen to the recorded the recordings there as well and we have had a few digital socials as well just to get a sense of how our members are doing in the pandemic and get some feedback on the different.

initiatives that the forum is uh planning can we move to the next page thank you so now i wanted to give you a very very brief overview of our worms events that you’re going to be seeing at our annual meeting so under our warm sponsor session there will be a panel discussion with minority women in operations research and management sciences and that will be on monday october 25th at 11 a m later in the day we will have a career.

uh coach discussing about how to deal with imposter syndrome perfectionism as well as the tendency that we have to compare ourselves against others and that will be on the same day at 2 45 p m and both all those times that are listed are pacific times our business meeting will be on monday october 25th at 6 15 pm uh since were not gonna be having the lunch on this year due to the pandemic were going to be hosting our award ceremony during the business meeting and it will be in a.

hybrid format right after the business meeting we will have our networking reception starting at 8 p m it will be in a very nice venue called the fifth rooftop restaurant and bar i heard that you can have a nice view of the disney park and if you have the opportunity to attend the conference in person feel free to stop by you may get an opportunity as well to watch the fireworks from there so if you would like to have a more diverse community and more equitable community in terms of gender representation please.

consider becoming a worms member if you’re not already one um i would really appreciate to have more male representation uh as men can be great supporters and advocates for women we are very active in different social media we have a twitter account we have a linkedin page a linkedin group and we also connect with our members through the informs connect so that’s all that i wanted to share um about our forum and thank you again aaron so much for this opportunity hopefully i didnt take too much time.

from this presentation not at all that was a wonderful uh introduction to to that particular forum so lets see oops all right so on to the uh todays presentation so just give me one second i hope you guys can see it okay all right so um todays um the topic for todays presentation is using ai to co-develop new product.

formulations and the future of flavor innovation and we are very pleased to have dr robin luigi um for this webinar robin is a very accomplished technical innovator with a track record of pioneering cross-industry multi-disciplinary research collaborations during her 26-year career as an industrial mathematical scientist at ibm research robin led technical teams in engagements with ibm and his clients co-authoring over 20 patent filings her project portfolio spans industry sectors including energy manufacturing supply chain logistics public safety and.

finance most recently she led a multi-year research collaboration with mccormack and company to establish a computational creativity system for ai enabled product development this work was featured in the media including the front page of a special section in the wall street journal robin is known for her service to the informs community in particular for spearheading new initiatives that benefit industry and academia she was the founding manager and lead evangelist for coin or the open source initiative for operations research software she spearheaded the informs professional.

colloquium now ecpm and she was the founding chair of the syngenta crop challenge in analytics robins contributions have been recognized with the informs impact price and the kimball medal she currently serves on the national academy of sciences board on agriculture and natural resources and advisory councils at the university of florida and cornell university robin earned her phd in mathematical sciences from the clemson university a quick note here we do have a well have a q a section or session after the presentation and you can see a q a button.

that you can use to submit questions and i strongly encourage viewers to do that option use that option to submit questions and robin will look go through those questions and ask them at the end of our webinar and we typically have a networking event for 30 minutes from 1 to 1 30 and most of the time we use it to continue with the q a session and uh so with that that’s it robin please take it over great hi everyone im just gonna give me just a sec while i share my.

screen we see that hows that reject very good okay excellent thank you ejection olga for the warm welcome and i appreciate all the work you do organizing these lunchtime at least in the east coast here in new york uh webinars theyre great to hear what everyones doing and im very im honored to present the work that my team did at ibm research so what im going to talk about today is using ai to co-develop new product formulations in the future of flavor innovation and.

this actually is um you know relies heavily on a presentation that i gave with my clients so i hope as weve done in some of the past webinars to give you that perspective from the client side as well so just a little bit of an outline about how the presentation is going to go is going to start with talking about the product development challenge you can see here in this picture um a soiree of different kinds of products that we see.

every day in the grocery store you know how do all these products get developed and what do product developers face when they try to create these new flavors then the second part of the talk is going to talk about computational creativity and you know why we care about it and then lastly im going to talk about a system that was developed and deployed.

using computational creativity to address the product development challenge so lets start off with the client and their problem so many of you may know mccormick this is mccormicks logo and you can see a few of their cons consumer products theyre over 130 year old company the largest flavor company in the world they have 35 brands and in those 35 brands over 17 000 products are sold in 160 countries around the world and to create those they have over 14 000 raw materials sourced from 85 countries so its big.

large and complex now you may know mccormick from your local grocery store you may even enjoy some of these products like you know i come from maryland and just love old bay seasoning but in addition to their consumer brands they also have a flavor solution unit so consumers only about 60 of their business labor solution is about 40 and theyre very different markets one theyre developing products for you the end consumers but in flavor solutions they have clients which are other restaurant uh services or cpg consumer product good companies that.

need a flavoring for their products so they have both consumers and clients and in their world of client delivery time is very much of the essence so this is a little bit about the complexity of the problem that the product developers are working on you know the group that creates these new flavors and new products is an outstanding research and development organization they have over 500 food science and flavor experts around the world and 20.

state-of-the-art technical senate centers in 14 countries and in fact morningstar has said that the mccormick research labs have become the best in the industry in their opinion and um the the team at mccormick is very humble but you know i they just have an outstanding group of people that are facing these challenges of how to create new products and for us as the consumers now its been said because mccormick is so big.

that its hard in the u s and the companies where they are to go 24 hours without ingesting something that has a mccormick product either directly or indirectly in it so theyre always trying to create new products for the changing tastes and one of the challenges in the flavor product development landscape is the changing consumer so todays consumers are younger right and theyre demanding new things.

you know for example when i was growing up you know we had strawberries you know in strawberry season and that was it now you know for me and my kids you see strawberries all year long you know the globalization of the food system has been extraordinary and these younger generations not not only want to taste all those ethnic flavors but they also care more about natural and sustainable flavors which is a challenge for the product developers so that’s the consumer but in addition the customers so that would be the the food and beverage services.

restaurants and other cpg customers you know theyre coming under more stress for cost management and economic value so its not just about flavor its not about great taste but its how to constantly reformulate these these products and maintain lower costs and low margins and then talent so it takes five to ten years minimum for someone after they graduate from college to become a master product developer they essentially go through an apprenticeship program and so theyre very rare um specialized and valuable employees and when they leave a company they take all that knowledge.

with them so there is a war on talent right now and i talked a little bit about the materials earlier but there are tens of thousands of raw materials to choose from for a product developer trying to create that next great best product let me give you an example in my kitchen i have two types of garlic you know i have the fresh garlic and then i have the powdered garlic.

in fact its only one type of garlic sold in my grocery store fresh and that’s that’s what i have but a product developer at mccormick literally would have 150 different types of garlic they would have garlic from asia and garlic from america and just like vidalia onions in georgia have a special flavor based on the soil there so do the different types of garlic based on where theyre grown and even the year they also have.

different formats so a powdered garlic but they have flaked garlic garlic granules and they have granules of different sizes so they have a huge variety of products and so there’s really a combinatorial explosions of possibilities that they have to deal with and today in the industry the only process that’s available to them is the stage gate process which is very iterative and time consuming and finally theyre constantly being asked to reformulate products for new regulations so different countries have different.

regulations you may know this from eating certain mac and cheeses that you know some of the coloring ingredients that are allowed in europe werent allowed in uh are are not allowed in europe are allowed in the u s so mccormick product developers have to be extremely agile and keeping track of all those relay regulations and adhering to them not to mention things like formulating for halal restrictions or manufacturability so overwhelming challenges in the product development landscape facing human product developers right now mccormick wants to address these.

because essentially they are a high-growth innovative cpg company so they have to continually reinvent and their vision is to elevate the capability of every global team member to the level of the best one so i mentioned before that um they had 500 scientists right well they have like 200 scientists in hunt valley but only say 12 in new delhi and its difficult to share the expertise of those developers across the globe.

and a junior developer may take for example 50 to 150 iterations to take a new idea from start all the way to ready for the shelf product formulation so whereas a senior member may take a third of that time so what theyd like to do is somehow elevate the capability of everyone in that diverse talent pool and right now i mentioned how slow and iterative the stage gate process was they want to increase that 300 3x moreover you may realize that new products on the shelf come and.

go quickly 80 percent dont really last more than a few years but there are some called icons which last for a long time like the oreo cookie or a philadelphia cream cheese but how does a product developer find one of those icons that is sticky in the market um so right now its a needle in the haystack so mccormick would like to make it easier for their product developers to find those icons because of course if.

it stays longer then that’s less innovation needed and so they want to make the product development faster better and more cost effective simultaneously and another trend that um that you probably very well with is the health and wellness trend in food products so they want to be able to enhance their ability to make healthy products for example.

there’s a lot of diseases linked to over consumption of sodium a lot of salt and products but by using certain flavors you can decrease the salt without decreasing the overall flavor experience so that in a nutshell is a summary of the product development challenges at mccormick and what theyd like to do so id like to switch now to talk about computational creativity so there’s a lot of talk and buzzword about.

using artificial intelligence for all sorts of business needs and in fact um you know ibm research has a long history of creating systems that push the state of the art in artificial intelligence you know from the deep blue system that beat the um worlds reigning chess champion gary kasparov to you know creating a computer system like jeopardy that uh like watson that played jeopardy.

and beat the reigning champions google for example you know beta system created a artificial intelligence system that you know was excelled at go so this is something that’s been going on in the computing world um is how to create systems that can think right and we typically have been looking at that as completing tasks or games but the challenge at mccormick is not creating a computer system that can think right its not just that a product developer has to think or just take known knowledge and.

apply it like in jeopardy where you you know solve a riddle its you know knowledge that’s known what they need is creative so it kind of begs the question of what is creativity and then what is computational creativity so id like you just to take a take a second and think you know whats your definition of creativity what does it mean to be creative does are you more creative than um you know then your family members how are you measuring creativity you know this is something that innately we know but actually is very difficult to pin.

down much like the definition of intelligence i like this definition of creativity its by bowden in 92 and its essentially a ability to come up with ideas or artifacts that are new surprising and valuable there are lots of different definitions of creativity but primarily they depend on something being novel and pleasing and often some other attributes as well so if we accept this definition of creativity we can also see that like.

beauty its in the eye of the holder like whats surprising to you may not be surprising to me so creativity is very subjective so for computational creativity you know were looking for a system that mimics human creativity so colton and wiggins had a definition of computational creativity as the phyllis philosophy science and engineering of computer systems which by taking on particular responsibilities exhibit behaviors that unbiased observers would deem to be creative so that’s a pretty good definition other people have different definitions that dont rely on unbiased observers but observers that are skilled in the arts.

and its really still an ongoing debate about how to define computational creativity but there is an association for computational creativity they’ve been having computational creativity if you havent heard of it events for the past 20 years and its a multi-discipline endeavor that is located at the intersection of the fields of artificial intelligence cognitive psychology philosophy and the arts because when i say creativity most people think about the arts painting you know drawing music you know dance lots of different different arts so there’s even been systems created that do jokes and puns so i thought i.

would just give you an example of a computational creativity system and since were talking about food um it involves fiber so what do you call a murderer with fiber and the answer is of course a serial killer so this joke this pun was created by a computer system called jape that was created by richie and bin stead in 1994 so you know we want to create.

a system to help mccormick and the product developers and its really important that it has that all-essential ingredient of creativity to address their needs so why would you want to think about creativity you know some people consider creativity to be the pinnacle of artificial of intelligence right um it also could be a test bed for your ai techniques just the challenge of intellectual challenge.

but in our case its also because of the applications so the team in um you know back in 2012 was working to create a computational creativity system and they decided to work in the area of food the way this came about i was told is because you know we have these grand challenges like beating a chess champion or um winning it watson you know watson winning at jeopardy well they decided wouldnt it be fun if we could win it.

iron chef you know the cooking challenge show but they quickly realized we didnt have the resources to build robotic systems so they decided to focus in on the recipes so they pursued computational creativity as an area of research with the domain of a culinary chef and creating recipes and they worked with the institute of culinary education well they were very successful and there was a light bulb moment when the chief science officer ahmed faridi who you can see here on christmas day.

was listening to npr radio the host all things considered robert siegel and there was an interview on his show of two people that one of the scientists at the what came to be known as the chef watson project on computational creativity for recipes with the culinary institute so they were working for the end user of a chef and he said it was a light bulb moment listening to love talk about.

computational creativity it was such a light bulb moment he had to pull his car over stop it was only a three minute segment but he said that segment fundamentally changed his vision for how mccormick was going to work um and so he started thinking about a lot of questions about the possibilities of a computational creativity ai system for mccormick because he had hired lots of.

contractors and consultants and they you know had told him that they didnt think it would work um so could we develop an intelligent platform to create product formulas or one that learns and gets better with experience because you know eating food is really an experiential thing you know and computers cant taste and computers cant smell so how could we develop a system that wasnt deterministic but one that that learned and could understand what a flavor experience meant uh for a consumer and how could they even predict what um.

people would like because end liking actually drives a lot of their business and when i use the word taste experience um let me just say a product developer cares not about just a taste at a point in time but they care about a taste throughout time so from when you first bite it that sensation while you’re chewing masticating smelling until you swallow that whole experience over time is what theyre designing products for and they actually formulate them differently so that you will experience.

different sensations at different points in time throughout that that period another another attribute was the human flavorists you know would they you know would they accept this um that the flavorists are very highly skilled and very proud rightfully so individuals you know would they see this ai as something that was helping them or would they perceive it a threat and so he got very excited and um reached out and partnered with ibm and it was he calls it a match made in heaven its really interesting mccormick is unusual and that they had 40 years of data.

so they estimate that theyll have a billion data points by the end of 2021 lots of chemistry food science data and over 400 000 flavor and food formulations that’s unusual a lot of people dont store that data over that such a long period of time in that industry and that that was what made this collaboration successful so what based on the conversation that what he heard on the npr talk hes very excited but there is there’s one.

problem is that chef watson was targeted for chefs you know you put in you put an ingredient in the cuisine and it helps you come up with new recipes product developers are night and day different from chefs in the view of mccormick the underlying approach that was used in chef watson was had some theoretical attributes when you come up when humans typically do.

creative acts well there are many different types of creativity in recipes were looking at combinatorial creativity where you’re taking lots of different ingredients mixing them in different proportions to come up with a new recipe that you try a lot of things and figure out which ones are good its sort of like a try and evaluate approach so when you want to do the trying that parts easy right its its kind of easy to just mix things together and come up something that’s that’s new the difficulty is coming up with something.

that’s new and good or new and useful so you need to have some sort of measure of good right some quantitative measure of whats good so in the chef watson project they used the idea of bayesian surprise so amazing surprise were essentially looking at the difference between a prior distribution and a posterior distribution and looking at that difference to see if what happened between the prior and the posterior was large then we say that’s very.

surprising but if theyre essentially the same we say its not surprising so this work did not match mccormicks view of how to proceed moreover the chef watson used the idea of the flavor hypothesis pairing in which case foods that share the same compounds chemical compounds go well together and should taste together so for example you may go to a restaurant and see chocolate leek cake you know.

they share similar ingredients so by adding a small amount of leeks to chocolate cake it really makes the flavor bigger and so that doesnt necessarily hold throughout the world mccormick is a global company so although weve been inspired um ahmed had been very inspired by the work with chef watson when it came down to it we had to start off with a different approach so the way we decided to approach the work.

had to do with thinking about the flavor space so this was very natural to product developers to think about exploring the flavor space and so we needed a way to digitalize the flavor space so we could explore it and of course being mathematicians and scientists very in natural way to do it is with a notion of distance so how do we take flavor and digitalize it so that we can explore the space and define define mathematically the space in a way that a computer can reason about it so.

we want to be able to generate suggestions new formula suggestions for product developers that taste great and were giving them not just the ingredients to use but the ratio of ingredients to use so in this schematic here the product developer may be looking in some area and they have a notion in their head that there’s a fruity space there’s a savory space there’s a sweet heat maybe.

a spicy heat there’s a creamy space in floral space the challenge in doing this is that even this characterization isnt standardized like there’s no triple e um description ieee you know standard for what these different um areas are let alone how to characterize them so let me talk to you about how the mccormick product developers interact with the system and then ill drill down into how the actual heart of the algorithms work so in the case of mccormick product developers its really directed creativity um we tackle the problem at the point.

where a new brief has been given to a product developer a brief is a description of what they should develop like i want a smoky a smoky hot sauce that hasnt doesnt have a taste of ash you know its very descriptive and they have to interpret that and the way it works is they come up with a seed formulation so a formulation is a list of ingredients and their relative proportions and that is the input to the system and then they can either revise that formulation or just simply go with that initial guess.

formulation and create a sample so by creating a sample they physically compound they physically make what that whatever it is that theyre making so in this case you know this spice the sauce and then they apply that to the protein lets say if that’s what theyre cooking and then they evaluate it so this is a hedonistic evaluation so human beings tasting it the gold standard for taste is is human for flavor experiences is a human and so that’s whats done and so they take those evaluations and based on that feedback they go back to.

the the lab and reformulate the revision and this this right here would be one iteration and so what they want to do is try to reduce that 50 to 150 iterations down to make it more quickly better and faster but once they get something that passes a certain hedonistic score typically a 6 5 to 7 then they have something that’s ready to be submitted either to the you.

know product development team at morgan at mccormick or to their client if its a flavor solutions business so this is the part here where the ai system engages with the product developer so lets drill into that a little bit so how the ibm ai for product composition system works so it works like an apprentice and its trying to learn the patterns and the best.

practices from the product developers through the historical data so we have a body of historical data here its the you know the materials the the raw ingredients and the attributes about them and the attributes depend on what kind of materials we have its the formulas in the repository and here its not just the winning formulas but its the formulas that didnt work because we want to be able to learn about what didnt work as well as what did work and then evidence of success so evidence success could be a formula getting.

picked to be compounded right to be evaluated it could be something that was sent to a client it could be something that was bought by a client it could be this the amount of time that its been being sold but we have some success evidence that we want to learn from so this body of information and we have five machine learning models there are substitutes complements application distance and success so.

first off were trying to learn patterns so substitutes are a kind of pattern so for example in your kitchen you may substitute flour for cornstarch these are functional substitutes they do the same thing but theyre a little different or lemon for lime slightly different taste but they both serve as an acid in the recipe we use machine learning models for complements to find complements complements are things that go together so basil and oregano often go together in mediterranean dishes and then application this wasnt originally in the system but the system was given a challenge of.

finding a cajun rice mix spice and the system did its magic and when it came back it came out with a suggestion that had no rice in it at all it was a very beautiful cajun seasoned salt recipe so quickly learned that the system needed to have some knowledge about what kind of application the user was working from because remember this is directed creativity the user has a goal theyre starting with that brief in mind and.

then finally id mentioned distance so were formulating the problem in terms of exploring things in flavor space so there’s a notion of distance and this is important because um you want to be able to sometimes be a little creative and sometimes you want to be a lot creative so we want to make sure were looking in space near where that initial seed was right because we have a target and so we use the idea of distance so we define distance between two.

formulas and formulas are tuples with the ingredient list and the relative proportions of those ingredients as the earth movers distance that’s one of the metrics we use which is a kind of optimal transport problem as you know derived from the wasserstein metrics so we have a notion of distance that weve derived and then finally success that’s what helps us choose between good formulas and bad formulas and using those five machine.

learning models and other data the ai system generates technically sound formulas that are hopefully good so one thing that happens is that the ai algorithm creates a lot a lot a lot of suggestions but we dont want to show those all to the users we want to focus them in we have a focus area that’s based on the target and the distance and then we try to generate.

we have to figure out which ones to show the users based on their goodness and so we try to cover that space well so for example if this is the focus area here um there may be some existing formulas that already exist so if depending on the application that were working on the users may not want to see those they may just be too.

similar to the existing product space and they might be much more interested in what exists in the white space so we also have a covering problem in here and i mentioned the idea of different levels of creativity for a product developer often theyre trying to not make something that’s wildly creative but their job is to make something taste better so for example how do you take an oreo cookie and make it taste better right how do you make a um a cheese snack taste better so there’s creativity involved into it but its more that theyre optimizing.

the flavors because the different mixes of ingredients can have the same flavor taste and other times they do actually want to be able to explore and find completely unusual ingredients so the system is trying to support not just one usage from the product developers but its been designed to support their whole life cycle of tasks that they do throughout their you know throughout their work and so our notion of creativity is.

actually a scale we can look at farther more increasing distance so so we built this system and it actually was deployed at mccormick in production many companies as you know are experimenting with ii but they tend to be proof of concepts whats really exciting about the system was that it got deployed into production and the product developers got to work with it so they had all the data was put into the system we had our five models that we learned.

from this design patterns of both the substitutes and complements which are building blocks of creating new uh suggested formulas and um that we then process to show the best ones where best means also were covering the space theyre targeted to the area theyre looking at and they have a high likelihood of success to show to the product developers so this is a little different view of the same the same system and um were very excited that it actually seems to be a value to.

mccormick and work you know one of the reasons or one of the things that’s very different about this project than any other project ive ever worked on is that the data that mccormick has of the formulas of the customers that it works with that its trying to improve its trade secret data you know this literally is the secret sauce so we worked to a large extent in encoded data so in other you know or type projects knowing the customers data better than.

the customer itself was one of the ways were able to add value but in this case we had to develop technologies and insights with the data being screened and then have you know later on find out how well it actually worked so there’s a different level of complexity but mccormick found that by using the system they were actually able to reduce their product development time in some instances by 70 percent and it became the basis for three flavors when they.

launched the one um platform so one product if you’re not familiar with it is the seasoning that goes on a bit of protein and vegetables so you can cook it in one sheet pan or one skillet and three of those products mccormick came out publicly and said were enabled by the ai system and so they are scaling this technology and in 2021 they hope to have it deployed across all their global labs and expect to reach that target of a 300.

percent increase in their innovations im very fortunate that mccormick and ibm went public with this work in february 2019 they published a joint press release to announce this work and the interest was really phenomenal had more than 100 emplacements including the front page of a special section on the wall street journals future of food there’s also a paper that was written for a food science journal if anybodys interested in youtube videos so ive talked a little bit about the challenge of the product developers and a little bit about computational creativity and the system that we.

created and to mccormick its not just about or the chief science officer mccormack its not just about using computational creativity for them but they really see this as revolutionizing the product development industry you know how product developers work because theyre being able to equip their product developers with improved starting points help them to more quickly develop their products because the system combines silos of knowledge from around the world its elevating.

everyones ability um through that 500 global scientists so global view is really essential to having one global r d rather than having a series of separate silos and that the kinds of datas that the product developers are able to have access to inherently through the a ai allows them to integrate sensory data you know from the start and the process so from mccormick its my hats off i think its a very visionary product project im been very grateful to be able to be a part of it.

so let me just tell you a little bit about what mccormick is telling the world about it is that theyre saying this computational creativity system um is giving them creative event advantage and not just for the food and the flavor but also to be able to re you can we think about reform having to reformulate products for cost for health and wellness for sustainability for carbon footprint they really think that this is going to be a new way of doing things and that its had an impact and how agile they’ve been for their customers.

you know i mentioned the flavor solutions portion of mccormicks business so theyre given um a bid they bid on say somebody wants a new strawberry flavoring you know all the flavor houses in the world would then compete if they can be the first person to come up with a good suggestion they can get feedback and that’s a competitive value advantage to them so actually the ability to do things quickly is is huge um i mentioned the talent war that was.

going on mccormick has had people very highly sought after product developers come to them and to be able to hire them because theyre working with ai and of course people who are leaders in their field often want to work with the latest and greatest technology so its actually has become a magnet for a better talent for them and um theyre using it to advance their flavor portfolio i dont know how many.

products in total now have been developed with the ai and the ai tools but the product developers dont call this ai they call it their innovation partner right so they dont see it as something that replaces them but they really see it as like the digital assistant to them and its helping them tremendously looking ahead um mccormick has always you know had this problem that there’s.

only so much self-space for their products so looking to e-commerce and the ability to want to mass customize their products um they just could never take that on its scale because they couldnt develop enough products quickly enough but a tool like the computational creativity tool allows them to do just that and theyre very excited about the opportunities for personalized flavor which is related to personalized nutrition so from the customers perspective you know.

theyre very excited about the future of flavor using ai and so ive talked about flavor and im very id love to talk more about flavor in the q and a but i just want to mention that flavor is only one application of product development you know a lot of the things that weve been talking about have analogs to fragrances or cosmetics.

or laundry detergents or lubricants or concretes right many consumer products have to be redesigned every year new new products or the same problems with having to reformulate for globalization reasons or different regulations or a product overall product or raw material is no longer available so you know its going away so you have to get something different and so the system that’s created at ibm research is actually more generally applicable and actually in 2019 it was used to develop uh the first ai.

fragrances they were sold by oh bhattacario and brazil that were launched in time for brazilian valentines day you know ibm had a partnership with simrice and so they were the um they were the suppliers to obadicario and it worked very much in the same way though i will say from my perspective i think the fragrances um you know are mostly olafaction to much greater descent than than flavor is you know flavor has a lot to do with other senses as well.

so i just like to say a big shout out to the phenomenal team at ibm research here who i had the pleasure of working with im no longer with ibm research but it was a joy and my honor to work with them and do this and thank you for the time today to let me tell you about the work hey thank you very much robin very unique application i hadnt really thought about it in that sense um.

and i get and beth if you can please open up the unmute the audience so if people have any questions they can ask those questions and and before before we start with any questions i just want to make sure that hey first of all thank you again robin for coming and making this presentation and for those in the audience we hope if you like these kind of presentations and again we have three series within the practice section that you will consider becoming a member.

of the practice session so if you have any questions please speak up i guess you are all unmuted now you can unmute yourself and and while we are waiting let me start off with a question also or two so robin you talked about you know right in the beginning you said flavor is star and strong demand and and so on and i guess my question is how do you evaluate.

or or judge a flavor is this it seems like a such a subjective term is it just because you’re able to be more creative is being more creative necessarily good in this business are you just by being more creative going back to you know flavors which have previously been discarded yeah i think there’s a lot of questions and that very good question you just asked rajesh yeah and and to me also its like its its so subjective.

you know i mean you know flavor you know humans are typically said they have five senses right you know we have sight look at what ai done has done with sites you know how um you know image processing can be done faster by a machine than a human right and that’s because weve been able to digitalize images right we we have pixelation and the rbg values so weve digitalized it and been.

able to work on the problem of giving computer site right um listening text you know your voice to text is something another problem that weve been able to to tackle because weve been able to digitalize it you know so what about ole faction and taste touch ill leave out because there are a lot of robotics in um in that but touch taste and olive action gustatory and.

olafaction are the least studied senses and the reason is because its just so complex in the area of flavors you probably know you may have been taught that you know your tongue has these little sections of sweet salt savory unami and its just its not correct its much more complex your perception of taste involves your eyes for example they did a study where they.

took two glasses of wine and asked people to rate them little did they know one was really red wine and the other was white wine with red food coloring in it and they evaluated the white wine as if it was a red wine you know proving that sight influence it you know when you eat that crunch right that affects your perception of taste.

and that is tactile right its also audio if you hear a crunch right so really i think flavor is about one of the most complicated domains to really work in because it involves it involves all of that so how does a you know how do we do this i think when most people hear about flavor work you think about yourself as being a chef you know you think about you and your kitchen and you know what are you going to make tonight and how is it going to.

taste or following a recipe that’s similar to a product developer because yes they do have a formula and ingredients but its also really different and ill give you an example lets say you want to create a potato chip right a flavoring potato chips i showed you that line of potato chips in that chart have you heard they cant have just one youve heard that like commercial right all right so there’s a whole science of.

um how do you create that response you know that response to have the next potato chip happens because of what your tastes is at the very end of that flavor experience journey so for example you may not have one garlic in it you may have multiple garlics you may have a very fine powder so that when you crunch the potato chip the fine powder coats your taste receptors on your tongue but see that washes away very quick so you may have to have a larger size granule so at the end of that flavor.

experience shell have it a product developer when theyre formulating a product and trying to decide whats good theyre thinking about that so its not its its not just the problem of a chef the chef has maybe a hundred and hundreds of ingredients product developers thousands completely different chef can think about like a very expensive rare balsamic vinegar product developer no way they have to think about being able to get those products and whats available.

so its very different and often a product developer is trying to make the flavor better like strawberry if i say the word strawberry you know what it tastes like right well there’s green strawberry there’s juicy strawberry there’s ripe strawberry there’s like a a whole lexicon of strawberry flavors that theyre trying to develop and improve on so its um its very different and they evaluate them on a hedonic scale um which you know theyre creating experiencing as humans the flavors that have a target and then theyre codifying in numbers you know how they.

rate that and that’s the input they give back so they have an agreed-upon way or a scale though it is subjective to say to say whats better so ive rambled a little bit rajesh i hope that a little bit answers your question what dishes are mute yes yeah thank you uh yes i wasnt muted so um absolutely you did answer the question and i know there are a lot of related sort of a small questions and i.

think you tried to address uh most of them before you go to ranganath um there are a couple of questions on the uh q and a panel i mean if you can open that panel sure sure so i see one question from irv uh so in case people cant uh see it or listening to the tape or it asks or bless the gasks if you’re.

applying this to other product um you still have to have a good data set like mccormick to know what are good products versus not right yeah i mean in these uh computational creativity systems these ai systems you got to have a good data set and i think that is gating the adoption by other companies um to applying these kind of techniques is because who else would have incurred the expense of keeping around product development.

data for 40 years right because were learning from what works and didnt so i think that’s a been a huge competitive advantage for mccormick but you need you need data and um you know were talking about you know the hundreds of thousands of product formulations you know and the 15 000 some odd raw materials i think the other thing about this data is you all as mathematicians know.

is that you know there’s data and there’s data right how many ingredients actually show up in any formula you know its more than 20 right so how many times is is your the raw material that you may be the best one for new application are you going to have data about that so its not just having data but its that kind of quality quality data and how might you try to capture that as a consumer product company um i think that the fact that this work has been successful for mccormick.

they have seen at least 70 percent a reduction in time they’ve proven all those consultants who said there’s no way that a system could be done wrong so i think that’s really sparking a lot of interest by other companies to start on that data of creating data sets for this and many people many people are also experimenting in their own right i mean the whole area of the digitalization of flavor is really really fascinating right now so great questioner the second question on the open q a is.

from bahidae abdi excuse me if i mispronounced that he said im curious if the process of cooking can be captured in these applications in many dishes the process of cooking with the exact same ingredients makes a very large difference in the final flavor so yes you’re absolutely right so i think what you’re pointing out buddy is that actually in many products there’s process steps like do you bake or do you boil you know do you distill do you know are you.

frying so the chef watson system actually worked more in that domain our current system the one that i was talking about was only combinatorial creativity in the sense that were looking at combinations of ingredients and the relative proportion and let me just say that that that was very challenging enough for us uh its still very much an ongoing research effort weve no by no means you know cracked um crack the nut on everything that mccormick wants but i think there are other people very much looking at the kind of applications that you’re talking about.

we also include process steps um you have a question yeah probably great talk you enjoyed it a lot and just a question about this process of interaction between ai part and the human beings like how did that happen the ai could come up with the formulation and then somebody would make that formula and the human beings would test it and evaluate and give some score and feedback to the ai is that how it worked that was something different yeah great great question i just want to.

say that um that one of the things that we were very fortunate to have in their champions at mccormick for this work um in hamed faridi was that he is a scientist you know by training hes and now the chief scientist hes not just a business person that but hes a scientist so some of the things that we struggle with.

when we deploy applications to end users he was on top of for example including the end users at the very beginning of the process and understanding change management so they actually hired a dedicated organizational psychologist to work with the product developers and the whole mccormick um you know to help them adopt to this tool because were not automating an existing process right there’s a lot of great work that does just that but what mccormicks.

doing is reinventing how their product developers work so changing it so there’s many different i talked about one use of the system you know there’s many many different uses and many different visualizations and tools but in what i talked about the product developer brings a seed so the starting point to the system and then they can play they can search inside the system they can play they can generate suggestions.

so you start with your starting point which is a recipe with ingredients and amounts and you can say generate suggestions based on this and so the system pops up like a dozen high qualified suggestions based on what it knows all the system knows is that seed actually the human knows a whole lot more because they’ve read the brief right they they know what direction they want to go in and so there’s still that gap there.

so the system can suggest things to them they can use it or not they can take that suggestion take half of it or they can reject it so there’s a loop there of how they interact and then you exit that loop when you decide that okay i want to taste this so then it gets compounded it gets applied in whatever way like if its a potato chip coating and then it gets tasted and then it goes back um so it gets evaluation so that.

information about what gets chosen how it got evaluated that goes back into the system so that we can continue to learn from it so did that answer your question wrong enough yeah no it does i was thinking in a particular application put something similar to jc carl that would have done similar things because also you are going to get some new suggestions have it okay did the advantage come from the fact that computer was suggesting many alternatives that the human beings would not think for their various biases yes and i should have made that point.

and i have like one of our earliest successes was there was a product developer who was doing pizza seasonings so im a human im thinking pizza im thinking italy mediterranean right so im thinking you know italian spices well the system came back and recommended cumin right and so the human had this bias of what does it mean to be mediterranean pizza the computer system didnt have the same bias and so it noticed that cumin generated a warmth and suggested that back to the user and the user thought wow that’s kind of interesting um.

so and there’s a lot of different examples like that and i think one of the interesting things is um that the human product developer has to decide if it makes sense or not so ive also you know in this case the product developer thought that was really cool tried it out and they liked it but there is a gate there where you have to win over the product developers trust to try something that’s outside their comfort zone i mean because theyre working under a.

pressure the time pressure in some instances they’ve come up with their own kind of go-to formulations like there’s hundreds of different cheeses so theyll come up with their own go-to on how to have a cheese flavor so what the system do will do is it doesnt it doesnt know that you know it knows what the rest of the product developers have tried over the years for cheese and it might suggest.

that so you have to hopefully have a product developer who will adopt or at least be open to trying what the system says otherwise it would never work thanks robin oh and i should mention wrong or not that the chief science officer said he personally almost contacted every single human product developer as a part of this project to you know keep them in the loop and make sure there was no unforeseen problems i think that’s pretty amazing commitment yeah thanks any other questions i dont think so there are no open.

questions anymore so um robin i think well stop here today then uh again thank you again for this wonderful talk and again for the audience uh please consider joining the practice group as well as the women in noah group as the case might be a video of this talk would be posted on the webinar home page you can go to the homepage and there is a list of past webinars and again if you have any.

questions or comments please send them out to me in terms of any news speakers or some speakers or a particular business on which you would like to see a talk just send any suggestions to me thank you again robin for a wonderful presentation and thank you olga for being a co-host on this one thank you very much rojas thank you robin it was a wonderful talk thank you olga okay thank you and thank you a bit bye.