Research Internship - Music playlist captioning

Paris, IDF, FranceIntern

Company Description

We are music and tech fans hailing from all over the globe, working to make Deezer the most personal music streaming service. From data scientists to tech experts, artists & labels specialists to marketers, and even in-house music editors, our team is spreading the love for music to over 180 countries. Supporting local and international artists and bringing them closer to their fans is our mission - we believe music is about diversity, multiculturalism and togetherness. Ready to join the team? We're all ears.

Job Description

Playlists are frequently created in music streaming platforms to organise tracks and artists or to provide batch, thematic recommendations to users [1-3]. While some playlists are created by music curators or users, others are automatically generated by recommendation algorithms. In both cases, finding a playlist title, description, and cover image, representative for the playlist content and appealing to users is challenging. Thus, the goal of this internship is to create a method to automatically caption playlists with both titles and descriptions. 

The problems of captioning or producing a short description of unstructured objects has been widely studied on some types of data such as images [4] or text documents [5]. Music-related data, on the other hand, has recently started to receive more attention. Choi et al. [1] tries unsuccessfully to generate full sentences to caption playlists with an end-to-to deep neural network. More recent efforts attempt instead to model the problem as a tag annotation task with more positive results [2]. Nonetheless, this latter approach is limited to a predefined set of tags without being able to dynamically include new tags, or longer or more expressive descriptions such as those created by a human editor. The state of the art model for music captioning [3] shows that relevant, human-like descriptions could be obtained for individual tracks with a multi-modal encoder-decoder with temporal attention. However, captioning playlists remains an open and challenging problem especially because playlists may be built around themes not (easily) deducible from the audio content only (e.g. decades, user activities).

The intern will be supervised by research scientists and research engineers from the Deezer Research team who provide practical and scientific help with the performed task. The intern is nonetheless encouraged to propose solutions and work autonomously. For data experiments, Deezer ensures cutting edge technology and appropriate calculus power.

References

[1] Choi, K., Fazekas, G., McFee, B., Cho, K., & Sandler, M. (2016). Towards Music Captioning: Generating Music Playlist Descriptions. In Proceedings of the 17th ISMIR. NY, USA

[2] Choi, J., Khlif, A., & Epure, E. V. (2020). Prediction of User Listening Contexts for Music Playlists. NLP4MusA 2020, 23.

[3]. Manco, I., Benetos, E., Quinton, E., & Fazekas, G. (2021). MusCaps: Generating Captions for Music Audio. In Proceedings of the 2021 IJCNN. Virtual.

[4] Chen, J., Guo, H., Yi, K., Li, B., & Elhoseiny, M. (2021). VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning. arXiv preprint arXiv:2102.10407. 

[5] Shen, S. Q., Lin, Y. K., Tu, C. C., Zhao, Y., Liu, Z. Y., & Sun, M. S. (2017). Recent advances on neural headline generation. Journal of Computer Science and Technology, 32(4), 768-784.

[6] Huang, F., Li, Z., Chen, S., Zhang, C., & Ma, H. (2020, October). Image Captioning with Internal and External Knowledge. In Proceedings of the 29th ACM International CIKM (pp. 535-544).

Qualifications

What we are looking for :

  • Master / PhD student with a background in Computer Science / Computational Linguistics / Applied Mathematics / Statistics.

  • Strong knowledge of natural language processing, applied machine learning and data mining

  • Good programming skills for data processing and experimentation (preferred python, but we are open to other technologies too)

  • Creativity and autonomy

Additional Information

Life @ Deezer HQ:

> Start-up environment with an at home vibe and outdoor space
> Kitchen stocked with free drinks and snacks daily
> Friday drinks & seasonal parties
> Gym access, plus yoga, pilates and boxing classes
> English and French language courses
> Hackathons & meetups

We are an equal opportunity employer