With 100 million users in its first two months, a growth rate faster than TikTok’s, ChatGPT’s integration with Microsoft’s Bing triggered both new consumer interest in the second-place search engine and a large reaction from Google.
After making some changes to Bing Chat due to some wild conversations users have had with the tool, a new wave of beta invites went out yesterday and we took it for our first spin. Here’s how the conversation went, and how advertising might be integrated when it comes out of beta.
TL;DR – We asked it to help with a trip to Punta Cana, Dominican Republic, and it suggested some great hotels in Ireland. Once corrected and presented with a more specific question, it provided very useful results. Unfortunately, the new cap on the number of interactions in one conversation cut the search short, which is a bit of a dealbreaker when the benefit of a conversational search is, well, a conversation.
“Sorry! They said I gotta go before I start speculating about how AI might bring about the apocalypse! I love you!”
– BingGPT, in my imagination
Given how well ChatGPT performs, we shouldn’t worry too much about early hiccups in an experimental software beta. That’s what betas are for. ChatGPT is, after all, able to pretend it’s Seinfeld or imagine what your dog might be dreaming about. It’s not surprising that in a new iteration, things might get a little weird during testing. Despite how the press loves to tear things apart when they find mistakes, PC World reported that 71% of users give the early beta a thumbs up.
Our search started off well, with BingGPT providing some immediately useful answers. Instead of rows of links, we get a friendly aggregated list of options with links to launch various sites. Much faster to read, and something that could work just as easily as an SMS, Voice or even WhatsApp interaction with Bing.
The most obvious integrations for advertising in the future here are:
- Added as a paid sentence in the conversation flow demarcated by either a different color text or the (ad) notation
- Included as a link option in the supplementary results in a different color
For the sake of brevity, I’ll leave out the screen where I ask for a resort in Punta Cana, Dominican Republic, but you can see Bing’s search activity looking up that destination. It handled that step well, but when I refined the question for a beach view, it returned results that were in Ireland. Nice hotels and country, for sure, but the wrong continent.
The Trident Hotel is in Cork, Ireland, but at least the Royalton Splash is in the right spot. Take me away, Calgon. Bringing up a map of Ireland was where it really went off the rails. When it is eventually working well, the integration of hotel reviews, photos and prices directly in the chat results will be a great feature. And another good spot for paid advertising integration.
When Bing’s error is pointed out, I feel bad for it. The sad emoji it sends me is a nice piece of humanistic conversational UX and a reminder that with AI search we aren’t in Kansas anymore.
Just like with ChatGPT, getting good results is really all about the prompt. If you ask a specific question, you’ll get a well-thought out specific answer. When I clearly state my intention in detail, Bing passes with flying colors, returning multiple travel providers that offer my specific departure and hotel request with pricing. This is exactly the kind of reply I was hoping for. From here, I could easily just click Transat, which is the lowest price option for the same hotel. As a WestJet fan, I might decide on that option instead for their great service.
Unfortunately, despite Bing asking me to keep going with the conversation by refining my travel dates and number of people in the party, it has to stop the conversation at this point to prevent any strange statements from being made. This rebooting of the conversation removes all memory Bing has of what we’ve been talking about, but highly likely given the lack of this limitation in its older sibling ChatGPT, that it’s just a kink to be worked out.
While the results come a little more slowly than we might be used to from dealing with chatbots built on simple logic trees, when Bing GPT works, it works very well. Better than traditional search results. There are interesting and clear advertising opportunities that Microsoft will surely integrate. While it may take some time for the platform to get running properly, it demonstrates an exciting new improvement in search.
Also published on Medium.
