Property descriptions at scale
Hypothesis
Expedia Group offers hotel partners a ready-made property description to use on our site. However, we were using structured content (CGS) to create the descriptions. This resulted in property descriptions that all looked the same and did not help hotels differentiate.
Our hypothesis was that by using ChatGPT, we could create unstructured descriptions that have more variability and granularity.
Value to partners: better representation of their property and the unique features/selling points
Value to EG: improve the shopping experience, opportunities for personalization, driving convergence and trust
Value to travelers: improved booking experience, enhanced trust in content, increased variety and quality of information presented
My Role: Content Design Director - prompt refinement, evaluating output
Approach
Our approach was to concatenate data from property reviews, extract key context from that data, and then generate a description using that context via a specific prompt.
I led this experience design effort with a content engineer and content designer on my team to:
Create a definition of good
Define metrics and grade output on eval metrics
Refine the prompt for accuracy, granularity, representativeness
Our learning metrics for the A/B test:
impact to conversion (Travelers booking hotel)
Impact to Avg booking value
Page bounce rate
Reduction / elimination of internal operation costs toward production/maintenance of custom content descriptions from hotel partners
Outcome
Initial tests were had positive reception with hotel partners who appreciated the variety and value. Actual text generation cost was deemed too high to continue test despite warm reception.
However, the work we did on the data architecture and prompt set the stage for new summary description features on the Traveler side of Expedia.