my name is Mark Regan and Im the product manager for retail AI and machine learning solutions so weve got a really packed agenda today retail has been an area of intense focus for Google over the past couple of years youll have seen from Thomas Koreans keynote on Tuesday at 7 out of the top 10 retailers are now building and innovating on GCP.
And were really proud to be working alongside them to be tackling a wide variety of problems for the retail industry, whether that be their supply chain. There are omni-channel commerce, experience or store operations. The list goes on, but for todays session. I wanted to focus on machine learning, use cases within retail that’s, the area that my team focuses on specifically and were really excited on behalf of many engineers and product managers to be
Able to share with you four new product announcements that are relevant to retail, and these are AI, enabled products that are truly transformative and. Im also really proud to be joined by three incredible. Companies that weve been partnering with closely over the past couple of quarters, that’s Nordstrom, the Walt Disney Company and Pitney Bowes theyre, going to come on stage and share with you their story of how they’ve been
using these AI products to transform and innovate within their own companies I think everybody here who works within retail lives and breathes the fact that were going through a period of you know transformation a difficult time in retail for sure and at the core of this the consumer is really driving a lot of the change their expectations have never been higher and that impacts everything across your value chain whether that be your supply chain the type of omni channel commerce experience you need to.
deliver even the store experience theyre expecting a personalized engaging shopping experience so I really think this is an exciting time but its also an incredibly difficult time and that’s what makes it so exciting for me because when I look at this retail value chain behind me I am convinced deeply that machine learning and AI will transform every single aspect of the you chain over the next five to ten years businesses will be built and torn down based on that innovation and that’s.
incredibly exciting but I want to be very concrete instead of focusing on the five-year or ten-year what is Google build today with our customers and our partners that you can start using to innovate within retail and so there’s four products that Id like to tell you about the first is contact center AI and visual product search recommendations ai and auto ml tables so let me dive straight in we all work within retail and we recognize that every customer.
interaction is a really precious opportunity opportunity to give a great experience to build loyalty to build brand and I think the call center is that unique place for someone as super stressed out has a big problem and they need it solved within Google we recognize that machinery and AI could provide a huge amount of value there and so weve built an AI agent that you can deploy inside of your call center so.
when your customer calls in the phone is answered immediately no more dial pad numbers that you have to press and you can converse with that agent in a natural language ask questions the agent can talk back they can add follow-up questions and move you through that decision process so that you can close out the case quicker and and have higher satisfaction now you may ask what if the issue becomes very complex not.
every issue is simple well we gracefully hand off the call to a support rep within your call center and they can close out the case so I think this is truly a transformative capability and were only at the beginning with regards to call center weve integrated with some of the biggest vendors in the market so if you’re interested call your contact center provider and ask them when theyre integrating with Googles context and our AI the second product is a visual product search weve been I think very fortunate within Google just.
experienced many consumer behavior changes over the period of our our companys history as an example we started off as a text-based search engine and we saw big emergence of voice being a key modality that people wanted to search on well today were seeing a new emergence that customers want to be able to search by taking photograph and were particularly seeing.
this within young Minette Millennials and certain verticals around apparel of fashion home goods and CPG the product that were announcing today and that were launching will enable you to build a custom computer vision model using your own product catalog so that your developer can deploy this feature within your mobile phone app enabling your customers to have unique experiences like you can see on the screen behind me the third product Id like to mention is recommendations AI we hear from customers and from the retailers that we.
work with consumers are demanding and engaging and personalized experience that’s what they want and they want it across every channel that theyre working on via online in-store through all the different marketing channels this is a hard problem and weve been working deeply on this within Google for years all of our products that you see search geo YouTube play have.
personalization and recommendation deeply bet embedded within it the product that were releasing today is the externalization of all of that research and capabilities so that you can train your own custom personalization recommendation model for your customers using your own data weve been running benchmarks against many competitors in the market for the past year and also against internal data science teams within retailers and across the board weve been seeing.
dramatic increases in these key KPIs that you see behind me the fourth product Id like to mention is called Auto ml tables within retail I think well all except the majority of the data were dealing with is structured data that means its lives in tables transactions product catalog pricing this is the fuel that machine learning models need to build models such as customer lifetime value propensity to buy a certain category how likely is this to be a fraudulent transaction and the list goes on there is dozens and dozens of machine.
learning models that can be built using data within a retailers org the table the product that were building that were announcing Auto Mall tables will enable your analysts and your developer as an engineer to build state-of-the-art machine learning models without having to write any machine learning code so my experience there’s usually ten or a hundred times more developer as an analyst in a company than there are data scientists so were trying to unlock the 99% Im really excited to see all the different use cases you can tackle here today so Id like to invite on our first.
speaker for today from Nordstrom John Shah who is vice president of technology leads a variety of different activities one of which is creating new engaging shopping experiences using computer vision so please join me in welcoming John to the stage thank you Mark last year I did a little experiment with my my teams at Nordstrom this is a product that we actually sell our website I show.
this product to my teams and ask them to come up with a set of search keywords and their objective was to use the search keywords to boost this product to the top of search result initially we just came up with a very simple keyword blue blouse and that’s what we got on the search result not even close that’s because this product is supposed to be wrapped off so we changed the term to a blue wrap top we got closer but were still missing some details so we had to add another keyword we made it we.
made it to a blue blue stripe wrapped up and now we got really close but the product were looking for is still nowhere near to the top of the search result also the experiment there were only two individuals were able to complete a task one of them work for the banging office she actually knew the exact brand name and father name and of course I worked the second one was an engineer for our search index scene the individual was able to get to the.
product detail page and reverse engineer a set of search keywords it goes something like this blue strap wrap top with side tie and double button cuff the whole point of this experiment is to illustrate its not about how well we can actually understand the search keyword function is over visual it has its own set of recoveries and taxonomy if I had if a customer has a product in front of them the best way to help them.
to find a part in our catalog is actually to visual search and this is why last year we start working on visual search there are two parts to video search step one is RP detection its all about giving a picture or frame of final video understand the object of interest once we understand the object interest we create a bounding box around it and.
then we want to the next step which is a similarity search and the my team and I put a demo together Ill see if I can play this so Northam is 120 year old fashion retail company emily starred as a shoe shop so my team and i we feel is only appropriate to do the visual search of a concept with shoe you can see we draw the bunion box we detect the object and then we send out we send the image to the back end and get the result back.
its really fast it took about two engineers maybe a month hold on a second yes thank you already initial an implantation of this was entirely open source space on the top you can see its a very typical machine learning pipeline we start with some simple images and then they use the open source model called yellow it stands for you only look once once we trend out the Domitian learning model we convert a to Worr ml.
and then were able to embed that model into our application so when we do an update detection it happens on device it never leaves the device that’s why we were able to really quickly identify the shoe on the bottom its a fairly complex pipeline we start with ingesting millions of product images into our machine learning framework and from there we use a feature extraction we get all the image features out of those images and then we run that through a.
clustering algorithm to put product in two different classes based on their video similarity and I run time once we detect the object we carved the object on the bounding box with send it to our app icon search API and then we have to go through the same process feature extraction identify which class or this image belongs to you and then do the product search we had two engineers work on this probably spend one month and it worked the hope a plan was built on top of GCP and we were able to get the accuracy of 80%.
what that means is if we saw this part of North on 80% time we can actually identify through image an 80% is actually not good enough we want to provide the best service and pass the experience to our customers however in order to close the gap the remaining 20% we actually had no idea how to get it down or how long its gonna take so I think was December last year we had the.
opportunity had the conversations with Google and then we form a partnership and we change our platform todays the table apart the object detection remained the same it was a pretty simple pipeline but as you can see the complexity that we had on the bottom part the video similar search was entirely replaced by Google vision paddle search we simply pass all our images to Google and acquitted Pradas at images and at runtime once we detect the object we send an object so our search hit Search API and then to.
Google vision for the search it just returns result this new design simplified our architectural design cut down our service cost and our accuracy was over 95% on day one so there are a lot of benefit switching over to this cloud enabled platform our engineers now can spend more time actually design user interface or better user experience what were gonna work on next is actually try to figure out who how to do a multi object detections and I found there.
were also going to enable the sequel this experience to all product categories thank you thanks John I think its incredible to see the innovation happening within Nordstrom and how this Gil helps you move quicker and get better model performance really exciting work its been great working with you next like to invite Kathy depalo whos vice president of engineering at the Walt Disney Company weve had a great partnership with Disney over the past couple of years and one of Cathys many responsibilities is building out a personalization and recommendation.
experience for the for the Disney customers online and in the parks hes welcome me in an inviting Kathy on stage [Applause] thanks mark so before we go forward I want to tell you a little bit about my team and where we sit in the broader Disney organization so we work on experiences that are both behind the scenes and guests facing and we are charged with creating products and experiences that bring our characters in our stories to life so Disneys guests have deep affinities for our brands and our.
characters and its our job to make our guests feel understood and well served whether theyre in our parks or in our stores now Disney has over 200 retail stores in North America as well as stores in Europe Japan and China we have over 600 retail locations inside of our parks and shop Disney is the e-commerce arm of the Disney Store on shop Disney we sell dedicated disney and disney parks merchandise as well as licensed.
product from our partners such as coach Dooney & Bourke and rag-and-bone the products include franchises from Marvel to Pixar to Star Wars and really a very broad variety of categories of product so everything from home goods to toys collectibles costumes etc so how do we help our guests find what theyre looking for this is where we decided to partner with Google on a data-driven approach so we were already showing recommendations on our product detail page which is where you get to if you click into an item on our website we wondered if we could do better using.
deep learning models that were personalized with real time user interactions so we worked with Google and trained a model using a years worth of anonymized user interactions and then we tested so we tested our old model versus the Google model now you can see the difference here the old model was keying purely off a product metadata which frankly can be a little confusing so the sir guests search for a sleep mask resulted in the items on the left which you might not really want to sleep in on the right.
hand side you can see the Google model figured out that it wasnt just about the mask it was what it was about the sleep and suggested much more appropriate items so the result of our test which we ran during our busy holiday season was pretty impressive across the board we saw increased engagement which is what we were optimizing for so that was good news but we also as the second-order effect saw increased revenue so we were encouraged.
by that and decided to try another placement on the Freak on the add to bag modals so once youve already clicked on an item and said yes I want to add this to my bag you see a pop up and in that pop up now youll see more items that you may like so again we tested Google versus the previous recommender you can see the example on the left hand side where again were keying off of the product metadata so it was a light-up costume and the recommender picked up on the light and suggested a bunch of.
lights whereas on the right hand side the Google model which was leveraging millions of past user interactions figured out that these are items that are frequently grouped together and bought together and so we got much better results heres another example where the user was looking for a red dress to added the red dress to their bag probably they dont want another red dress so the Google model figured out that users were more likely to purchase things like shoes and accessories if they had already added a dress to the cart so in this case we had not.
previously been showing recommend a Asians in that placement and so when we ran our tests we actually ran an ABC test we tested the Google recommender versus the old recommender versus no recommendations at all and got some interesting results the Google recommender did well the no recommendations test case actually did better than the old recommender and so working with Googles engineers we figured out the reason for that which was on certain mobile devices and.
certain web browsers the addition of the recommendations at the bottom of that mobile Moe the modal actually pushed that add to bag button off the screen so that was a good lesson learned for us which is its not just about the model quality it can be equally important to focus on the placement and the design of how that models presented to the users so speaking of lessons learned and takeaways number one you guys probably all know its all about the data.
when we started out our data was woefully unready for use in machine learning and frankly I think the only way to get it ready was to start using it to build models and along the way we discovered all sorts of interesting things about our data that had nothing to do with ml but were actually useful to the business placement matters so the type of model the design of how it is pulled onto the page is really important.
testing you really cant know until you you get it out there and start comparing live not just offline models and then finally I think as John mentioned leveraging Googles expertise was really key for us on this it allowed us to not have to worry about things like scalability getting that last 10 to 15 percent of accuracy that we were looking for and to be able to really put out a.
high quality product on day one so happy ending to our story were currently running both of these models live in production and although were no longer testing we are seeing increased average order value units per transaction and revenue so going forward were continuing to work with Google and seeing many more opportunities for where recommendations and cloud-based ml can have a great impact for us and our ability to serve our customers thanks very much Thank You Kathy that’s fantastic its great to see how weve been able to partner with Disney.
to create new experiences on the shop Disney size leveraging a lot of our research that weve pumped into recommendation personalization and deliver that as a managed service that you can start using today to deliver personalized and recommended experiences for your customers next up Id like to invite Olga like Innova who is the chief data and analytics officer for Pitney Bowes again another fantastic partnership for the cloud AI team and Pitney Bowes please join me in welcoming Olga [Applause] hi Im going to switch the slides and it works fantastic so let me tell you what.
Pitney Bowes is Pitney Bowes is a global technology company that helps our customers always providing solutions around Commerce and we serve a large number of customers from small medium businesses to Fortune ninety percent of Fortune 500 customers companies but one of very important segments of the customers for us are retailers our fastest-growing commerce services business serves more than 700 retailers.
and we are the number of retailers growing every day so what do we do for these retailers well when Mark was showing you the slide that shows retail value chain there was little little areas there that said logistics fulfillment and delivery and that’s what we do we help retailers with e-commerce with logistics fulfillment and delivery and Mark also talked about increasing customer expectations so if you think about your experience in on e-commerce.
side and when it comes to ordering some things what is your expectation I want something fast and I wanted free and if I dont like what I get I want to return it free and so this fast and free and please tell me every single moment of my existence where my packages today because I like it like if I could have an uber app that shows me well how my package comes to me that would be really cool Wow that’s expectation is actually not that simple to accomplish for retailers so.
this is why Pitney Bowes come in and we we provide Play forms that enable us through the scale help retailers to accomplish this fast and free delivery and return domestically and internationally that’s what we do by the way if you dont know 90 percent of us when you go to the e-commerce site and you dont think that what you see is fast and free enough for you you’re going to live which means is that a retailer just to lose you as a customer and you may never come back that’s really bad right so that’s what.
we want to avoid this is just an example of some of the retailers Im happy to say that my our Nordstrom colleagues are part part of it today we are going to talk about very specific business part of Pitney Bowes business for retailers and were going to talk about cross-border business its a selling internationally its a very big opportunity for retailers in the United States about 50% of people of people who shop online shop internationally it means that I would buy from China or UK sometime this year if you are in.
China 79 percent of consumers shopping internationally so its a really great market opportunity for retailers but its not that simple to get into this business there are a lot of complexities so this is why Pitney Bowes come in Im not going to go through this rather complex slide but Im just going to mention couple of things and I will explain why we are so excited about our ml so this business of cross border.
shipping is impossible for us to do if not by day if it wasnt driven by data ai and ml the efficiencies that are necessary to provide the good value to retailers demand a high skill ability in the area of AI nml and applying it in many many areas we are going to look only at one of them today fraud and this is where we started just in outer normal tables but our vision is to work continue working with Google so we continue scaling our abilities just give you a couple of examples right so you you come to the.
website you got to the localized website that we can help help retailers to create but now you’re going to be quoted how much is going to cost the ship a particular item and you’re just understanding what this item is sometimes is not that simple it that yes for retailers with a very good catalog its easier but if its eBay seller its not so easy there is a lot of natural.
language processing and a lot of AI nml behind it just I dont to understand what is going to ship cost you to ship there are custom optimization customs optimizations that we can do so we can predict whether we need to check this particular item opens a box and check it before we ship but things like that every single step the efficiencies are driven through data and very advanced.
analytics part of it what the service is Pitney Bowes provides these retailers is international payments which the risk has a stone set of fraud detection but also we do additional level of detection on international orders there are there is a patterns that you can identify in the behavior or based on the history of a particular seller or the history of where how are the skills are shipped to a particular address even if we didnt detect as a fraudulent transaction through the through the credit card you.
could sometimes see a patterns by looking at multiple credit cards by going the same to the same address so there are a lot of things that we do in this area to understand whether the particular order is is fraudulent or not now this is where we started experimenting with outer ml tables just just to give you an example of why it is really important to look at the fraud fraud rates rise significantly constantly this is a very dynamic and very aggressive part part of fraudulent attempts and malicious attempts to.
attack and if you look at apparel cosmetics and perfumes jewelry consumer electronics all these areas that our retailers ship internationally are subject to to fraud and and so when we look at this problem of identifying fraudulent international orders there are three goals that we want to accomplish the first one obviously do not let fraudulent orders to go through that’s the most important thing so identifying correctly all the.
all the fraud is important but also very very important is not to have false positives and one of the big motivators for not having false positives is we really do not want to affect experience for the good consumers right its its not a great consumer experience and it is not serving our retail customers well so we do have a team that goes and looks through orders that were identified as fraudulent especially if our level of confidence is not high so this team.
hopefully is going to take care of the consumer also we need to take care of this team right its a manual process we want to reduce manual process as well so looking at how we solve this problem we started with a rule-based system and this rule-based system helped us to decide which orders are accepted off or held for review its a third party rule-based system it worked for us okay for period of time but the rules became increasingly complex really difficult to.
maintain and too slow to adopt to rapidly change infra-red environment so we started by augmenting this rule-based system with email based models and its a constellation of models the one of them is based on a G boost and it trained on fraud labels to score to score the transactions and that has been very successful for us when when when we were implemented it fraud loss rates reduced by 49 percent and we review efforts reduced by 14 percent but so so whats the problem well we over the years developed a lot.
of expertise in running production models against ecommerce transactions we have a really good team of data scientists and ninja engineers who developed a proprietary framework whereas these models run we have 20 mil 20 milliseconds response time on all the calls with SAS we run multivariate testing we monitor we monitor these models very closely because every mistake in the model actually direct direct hit is a on definitely on our margins and our profitability and.
profitability for our retailers but it takes time and it takes effort to develop the models and then deploys them and manages a lifecycle of the model what if we could multiply our ability to bring iin ml competences to our business what would it do for us I think that its a tremendous impact for us and so in this particular model a we wanted to.
prove that we could use outer ml table to just automate and speed it and get it to the simplicity of business analyst or our engineer to just create the model and run with it the deployment is a snap without elegance it is now it just all there automatically happens so we knew that would be a huge boost so that’s what we did in addition we were not.
necessarily happy with the precision of them of the our model we had false positives that we wanted to eliminate and knowing we were very happy to work with Google to make sure that we can take advantage of this Kegel competition winning kind of quality of the models and see what is going to happen if were going to apply them to our problem and that’s what we did so we used our terminal tables we were one of the alpha.
alpha users of this technology we use it with binary classification and we tested it o is and without feature engineering because a big value proposition of outer ml table is that you dont didnt need to do future engineering the promise of it is that you could you could get the same level of performance from the model even without feature engineering and when we did that we proved this is the case we took the raw data with very minimalistic things that we had to do to to remove some PII information but when we ran out ml table against it the.
performance actually was better than the models that we were using and not only that the whole ability to reduce time to develop and deploy model from months to actually now its just days as outer ml is getting into the next beta stage and going bid its a really powerful powerful tool that we hope to make available to our data engineers and.
advance business and business analysts to democratize AI to democratize AI and make it make it making even bigger bigger impact on our business and the way we serve our retailers thank you [Applause] Thank You Olga so weve covered a lot of content today hope you’d agree that some of these products that were bringing to market are really exciting for the retail industry we really enjoy working deeply with retailers and retail technology partners in bringing this technology into your business so just to quickly recap context NRA is using some.
of our cutting-edge understanding of voice understanding voice synthesis and real-time production engine to deliver and automate it experience your calls call center this is truly a breakthrough capability were also using our cutting-edge computer vision technology to enable you to take photograph and search by image and our recommendation ai product is using using all of the research that weve pulled from YouTube and Google Play so that you can deliver a per user personalized recommendation experience thatll drive on your business KPIs and then finally.
automotive tables can tackle a whole variety of business problems for you whether that be forecasting how much product you now need to have in every store or whether it is detecting fraud or understanding the likelihood of a customer to buy a specific product so if youve got any questions and you’re interested in talking about this further Im gonna be on the side of the stage but Id also recommend that you take down this email alias here which actually connects you to product managers engineers and people inside of Google that would love to talk to you.
more deeply I will say that this is just the tip of the iceberg we have a big team focusing on retail and its our commitment to you that were going to be bringing more AI capabilities to the retail industry over the next year to 3/4 into the future and were really looking forward to working alongside you such that you choose GCP as being your.
platform of choice leveraging his capabilities and all of the other stuff that weve talked about at this conference so Id like to thank the speakers for joining me on stage today and thank you all for for joining us.
Leave a Reply