DrupalCon Amsterdam 2019: I Think It’s a Kitten Wearing a Bowtie? Drupal Caption Suggestions

Nov 01, 2019

Laura Johnson, Boban Stanojevic
Myplanet, Toronto, Canada

As developers and content editors, we often feel the urge to take shortcuts when entering image captions. It’s time consuming! (And weirdly kind of hard sometimes to come up with the right words.) But what if you had an option to have your caption automatically generated with AI? Well that would be very nice, indeed.

In this session, we’ll showcase a proof of concept using an AI-driven image captioning API from IBM to suggest image captions when uploading images to the media module. AI image captioning could be extremely helpful, but it’s never going to be as good as human image captioning (and often gives hilariously bad suggestions), which is why our POC includes a user prompt to review the caption before saving.

We’ll demo the proof of concept and show the code behind it, talk about the possible benefits to accessibility using this approach, as well as possible downsides, and we’ll give an overview of the elements behind the AI model and talk about considerations when choosing an AI model for integrations. We’ll also talk about how the same method can be adapted as a CKEditor plugin for inline images.

What you’ll learn:
How we integrated the AI Captioning API with Media
Accessibility benefits and possible pitfalls
Elements behind the AI Captioning model
Considerations when choosing an AI model
Possible variations and improvements to make in the future

Hi! I'm Karim Boudjema, the developer and site-builder of this site. I'm currently working as a freelance Drupal developer.

I was wondering how could I giving back to the Drupal community all what it gave to me during the last 10 years, that's how the idea of this site came to me ... (read more)

Hi! I’m Santiago Rico, the web designer and themer of this site.

When Karim proposed me to design and theme this site for the Drupal community, I accepted immediately because the community is the main reason why I love Drupal so much.