Implementation of Bidirectional Encoder Representations from Transformers in a Content-based Music Recommendation System for Digital Music Platform Users
Fadil Abdillah Suyudi (1), Muhammad Ariful Furqon (2*), Qurrota A'yuni Ar Ruhimat (3) (1) Universitas Jember - Indonesia
(2) Universitas Jember - Indonesia
(3) Universitas Jember - Indonesia
(*) Corresponding Author
Received: July 24, 2024; Revised: September 18, 2024
Accepted: December 18, 2024; Published: August 31, 2025
Abstract
Digital music platform users today have unlimited access to millions of songs from various genres and artists through music streaming services. However, with so many music platforms available, users often need help finding songs that suit their preferences. This study presents a music recommendation system that utilizes lyrical analysis to provide users with relevant song suggestions based on selected lyrics. The system employs a two-pronged approach: the Term Frequency-Inverse Document Frequency (TF-IDF) method for initial feature extraction and the IndoBERT model for advanced contextual representation of song lyrics. A dataset of 8,944 Indonesian language songs was compiled using scraping techniques from various sources. The recommendation process is driven by cosine similarity calculations between the lyrics of the selected songs and the entire dataset, enabling the identification of songs with similar themes and messages. Model evaluation through a five-fold Multi-Class Cross-Validation (MCCV) approach yielded promising results, indicating high precision, recall, and F1 scores. The study results show that the system built can provide recommendations with good precision performance with Precision@k values varying between 0.7965 to 0.8371, Recall@k values ranging from 0.8017 to 0.8204, and F1-score@k values varying between 0.8083 up to 0.8190. Overall, the model shows strength in providing accurate recommendations and good performance stability
http://dx.doi.org/10.55981/jet.660 Keywords
References
P. Wikström, The music industry: Music in the cloud. John Wiley & Sons, 2020.
A. Dutta and D. K. Vishwakarma, “Personalized music recommendation system based on streamer streaming trends,” 2021 12th International Conference on Computing Communication and Networking Technologies, ICCCNT 2021, 2021, doi: 10.1109/ICCCNT51525.2021.9580113.
M. Schedl, P. Knees, B. McFee, and D. Bogdanov, “Music recommendation systems: techniques, use cases, and challenges,” Recommender Systems Handbook: Third Edition, pp. 927–971, Jan. 2022, doi: 10.1007/978-1-0716-2197-4_24.
S. Mutturaj, S. B, S. P, S. Beldale, and S. V, “A survey on hybrid recommendation engine for businesses and users,” International Journal of Information Engineering and Electronic Business, vol. 13, no. 3, pp. 22–29, 2021, doi: 10.5815/ijieeb.2021.03.03.
J. Van Balen and B. Goethals, “High-dimensional sparse embeddings for collaborative filtering,” The Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021, vol. 2, pp. 575–581, 2021, doi: 10.1145/3442381.3450054.
N. Ula, C. Setianingsih, and R. A. Nugrahaeni, “Sistem rekomendasi lagu dengan metode content-based filtering berbasis website,” e-Proceeding of Engineering, vol. 8, no. 6, pp. 12193–12199, 2021.
M. Kaminskas and F. Ricci, “Contextual music information retrieval and recommendation: state of the art and challenges,” Comput Sci Rev, vol. 6, no. 2–3, pp. 89–119, 2012, doi: 10.1016/j.cosrev.2012.04.002.
A. Vaswani et al., “Attention is all you need,” Adv Neural Inf Process Syst, vol. 2017-Decem, no. Nips, pp. 5999–6009, 2017.
H. Anushree, “EFFICIENT recommendation system,” vol. 12, no. 2, pp. 491–500, 2021, doi: 10.34218/IJARET.12.2.2020.047.
M. H. Abdi, G. Okeyo, and R. W. Mwangi, “Matrix factorization techniques for context-aware collaborative filtering recommender systems: a survey,” 2018.
B. Hawashin, A. Mansour, T. Kanan, and F. Fotouhi, “An efficient cold start solution based on group interests for recommender systems,” ACM International Conference Proceeding Series, 2018, doi: 10.1145/3279996.3280022.
Y. Deldjoo, M. Schedl, P. Cremonesi, and G. Pasi, “Recommender systems leveraging multimedia content,” ACM Computing Surveys (CSUR), vol. 53, no. 5, pp. 1–38, 2020.
Y. Zhang, “Music recommendation system and recommendation model based on convolutional neural network,” Mobile Information Systems, vol. 2022, no. 1, p. 3387598, Jan. 2022, doi: 10.1155/2022/3387598.
M. Schedl, “Deep learning in music recommendation systems,” Front Appl Math Stat, vol. 5, p. 457883, Aug. 2019, doi: 10.3389/FAMS.2019.00044/BIBTEX.
M. G. Galety, R. Thiagarajan, R. Sangeetha, L. K. B. Vignesh, S. Arun, and R. Krishnamoorthy, “Personalized music recommendation model based on machine learning,” 8th International Conference on Smart Structures and Systems, ICSSS 2022, 2022, doi: 10.1109/ICSSS54381.2022.9782288.
A. Elkahky, Y. Song, and X. He, “A multi-view deep learning approach for cross domain user modeling in recommendation systems,” WWW 2015 - Proceedings of the 24th International Conference on World Wide Web, pp. 278–288, May 2015, doi: 10.1145/2736277.2741667.
H. Zhu, Y. Niu, D. Fu, and H. Wang, “MusicBERT: a self-supervised learning of music representation,” MM 2021 - Proceedings of the 29th ACM International Conference on Multimedia, pp. 3955–3963, Oct. 2021, doi: 10.1145/3474085.3475576/SUPPL_FILE/MM21-FP2184.MP4.
M. A. Khder, “Web scraping or web crawling: state of art, techniques, approaches and application,” Int. J. Advance Soft Compu. Appl, vol. 13, no. 3, 2021, doi: 10.15849/IJASCA.211128.11.
C. M. Dupin and G. Borglin, “Usability and application of a data integration technique (following the thread) for multi- and mixed methods research: a systematic review,” Int J Nurs Stud, vol. 108, p. 103608, Aug. 2020, doi: 10.1016/J.IJNURSTU.2020.103608.
M. Vystrčilová and L. Peška, “Lyrics or audio for music recommendation?,” in ACM International Conference Proceeding Series, 2020, vol. Part F162565, pp. 190–194. doi: 10.1145/3405962.3405963.
D. C. Corrales, J. C. Corrales, and A. Ledezma, “How to address the data quality issues in regression models: a guided process for data cleaning,” Symmetry 2018, Vol. 10, Page 99, vol. 10, no. 4, p. 99, Apr. 2018, doi: 10.3390/SYM10040099.
K. Dekker and R. Van Der Goot, “Synthetic data for english lexical normalization: how close can we get to manually annotated data?,” in Proceedings of the Twelfth Language Resources and Evaluation Conference, 2020, pp. 6300–6309.
D. J. Ladani and N. P. Desai, “Stopword identification and removal techniques on tc and ir applications: a survey,” 2020 6th International Conference on Advanced Computing and Communication Systems, ICACCS 2020, pp. 466–472, Mar. 2020, doi: 10.1109/ICACCS48705.2020.9074166.
D. Khyani, B. S. Siddhartha, N. M. Niveditha, and B. M. Divya, “An interpretation of lemmatization and stemming in natural language processing,” Journal of University of Shanghai for Science and Technology, vol. 22, no. 10, pp. 350–357, 2021.
M. O. Sanjaya, S. Bukhori, and M. Furqon, “Virtual assistant for thesis technical guide using artificial neural network,” Indonesian Journal of Artificial Intelligence and Data Mining, vol. 6, no. 2, pp. 188–196.
Y. Zhang et al., “DIALOGPT: large-scale generative pre-training for conversational response generation,” Proceedings of the Annual Meeting of the Association for Computational Linguistics, pp. 270–278, 2020, doi: 10.18653/v1/2020.acl-demos.30.
T. Maheshwari, T. N. Bhaveshbhai, and M. Halder, “The power of visual analytics and language processing to explore the underlying trend of highly popular song lyrics,” Engineering and Applied Science Letters, vol. 4, no. 3, pp. 19–29, 2021.
Y. Song, S. Dixon, and M. T. Pearce, “A survey of music recommendation systems and future perspectives,” Proceedings of the 9th International Symposium on Computer Music Modeling and Retrieval (CMMR), no. June, pp. 395–410, 2012.
U. Ungkawa, D. Rosmala, and F. Aryanti, “Pembangunan aplikasi travel recommender dengan metode case base reasoning,” Jurnal Informatika, vol. 4, no. 2, pp. 57–68, 2013.
N. Gali and V. Tiwari, “Speech and lyric-based doc2vec music recommendation system,” International Journal of Engineering Research & Technology (IJERT), vol. 9, no. 8, pp. 67–70, 2021.
S. Ebiesuwa and T. Awoniyi, “Analysis of nigeria’s top ten song lyrics using natural language processing techniques,” International Journal of Computing and Digital Systems, vol. 16, no. 1, pp. 1–9, 2024.
A. W. Qurashi, V. Holmes, and A. P. Johnson, “Document processing: methods for semantic text similarity analysis,” INISTA 2020 - 2020 International Conference on INnovations in Intelligent SysTems and Applications, Proceedings, Aug. 2020, doi: 10.1109/INISTA49547.2020.9194665.
J. Wang and Y. Dong, “Measurement of text similarity: a survey,” Information 2020, Vol. 11, Page 421, vol. 11, no. 9, p. 421, Aug. 2020, doi: 10.3390/INFO11090421.
B. V. Kartika, M. J. Alfredo, and G. P. Kusuma, “Fine-tuned indobert based model and data augmentation for indonesian language paraphrase identification.,” Revue d’Intelligence Artificielle, vol. 37, no. 3, 2023.
W. Wongso, H. Lucky, and D. Suhartono, “Pre-trained transformer-based language models for sundanese,” J Big Data, vol. 9, no. 1, pp. 1–17, Dec. 2022, doi: 10.1186/S40537-022-00590-7/TABLES/6.
L. Tunstall, L. Von Werra, and T. Wolf, Natural language processing with transformers. “ O’Reilly Media, Inc.,” 2022.
S. Wehnert, V. Sudhi, S. Dureja, L. Kutty, S. Shahania, and E. W. De Luca, “Legal norm retrieval with variations of the bert model combined with tf-idf vectorization,” Proceedings of the 18th International Conference on Artificial Intelligence and Law, ICAIL 2021, pp. 285–294, Jun. 2021, doi: 10.1145/3462757.3466104.
J. E. Ewusie, C. Soobiah, E. Blondal, J. Beyene, L. Thabane, and J. S. Hamid, “Methods, applications and challenges in the analysis of interrupted time series data: a scoping review,” J Multidiscip Healthc, vol. 13, pp. 411–423, 2020, doi: 10.2147/JMDH.S241085.
Q. C. Song, C. Tang, and S. Wee, “Making sense of model generalizability: a tutorial on cross-validation in r and shiny,” Adv Methods Pract Psychol Sci, vol. 4, no. 1, Mar. 2021, doi: 10.1177/2515245920947067/ASSET/IMAGES/LARGE/10.1177_2515245920947067-FIG3.JPEG.
V. R. Revathy, A. S. Pillai, and F. Daneshfar, “LyEmoBERT: classification of lyrics’ emotion and recommendation using a pre-trained model,” Procedia Comput Sci, vol. 218, pp. 1196–1208, Jan. 2023, doi: 10.1016/J.PROCS.2023.01.098.
A. Elbir, H. Bilal Çam, M. Emre Iyican, B. Öztürk, and N. Aydin, “Music genre classification and recommendation by using machine learning techniques,” Proceedings - 2018 Innovations in Intelligent Systems and Applications Conference, ASYU 2018, Nov. 2018, doi: 10.1109/ASYU.2018.8554016.
C. McDonald, A. E. Foster, and P. Rafferty, “Playlists and genre: the role of music genre in spotify’s playlists,” Journal of Documentation, 2024.
Article Metrics
Metrics powered by PLOS ALM
Refbacks
- There are currently no refbacks.
Copyright (c) 2025 National Research and Innovation Agency
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
JET is published by National Research and Innovation Agency (BRIN) - formerly LIPI
Copyright of BRIN :: p-ISSN:1411-8289, e-ISSN:2527-9955 
Email this article 
