Pengaruh Few-shot Learning pada Kinerja LLM untuk Ekstraksi Entitas Iklan Lowongan Kerja

Alvalen Shafelbilyunazra; Didik Dwi  Prasetya

doi:10.55382/jurnalpustakaai.v5i2.1069

Authors

Alvalen Shafelbilyunazra Universitas Negeri Malang
Didik Dwi Prasetya Universitas Negeri Malang

DOI:

https://doi.org/10.55382/jurnalpustakaai.v5i2.1069

Keywords:

Ekstraksi Entitas, Few-shot Learning, In-Context Learning, Large Language Model (LLM), Prompt Engineering

Abstract

Ekstraksi informasi dari teks tidak terstruktur, seperti iklan lowongan kerja, merupakan tantangan besar. Pendekatan tradisional berbasis fine-tuning membutuhkan dataset berlabel masif dan sumber daya komputasi tinggi. Sebagai alternatif, Large Language Model (LLM) dengan In-Context Learning (ICL) menawarkan efisiensi. Penelitian ini menginvestigasi pengaruh few-shot learning, khususnya variasi jumlah contoh (k), terhadap akurasi LLM dalam ekstraksi entitas dari iklan lowongan kerja berbahasa Indonesia. Menggunakan model Gemini, eksperimen dilakukan dengan skenario zero-shot (k=0) hingga few-shot (k=1, 3, 5, 10, 20). Setiap skenario dievaluasi lima kali menggunakan Monte Carlo Cross-Validation, dengan metrik Presisi, Recall, dan F1-Score. Hasil menunjukkan korelasi positif antara jumlah contoh dan akurasi, namun dengan point of diminishing returns. Peningkatan kinerja drastis terjadi pada 1-5 shot, dan performa mencapai kejenuhan setelah 10 shot. Model cenderung memiliki Presisi lebih tinggi daripada Recall, memprioritaskan kebenaran ekstrak. Studi ini menyimpulkan bahwa strategi prompting optimal memerlukan keseimbangan akurasi dan efisiensi, merekomendasikan 5-10 contoh untuk sebagian besar aplikasi. Temuan ini memberikan panduan praktis untuk optimalisasi penggunaan LLM dalam ekstraksi informasi.

Downloads

Download data is not yet available.

References

Fareiz Aulia Firman, Dr. Vip Paramarta Drs., MM, Rocky Fransiskus Budiman, Yuliani Salewe, and Karlis Karlis. 2023. Fungsi SDM Sebagai Pemain Strategik Manajemen Modal Insani dan Manajemen Talenta. Journal of Creative Student Research. 1(3): 289–303.

T. Ghorpade. 2024. Online Job Portal. Gurukul International Multidisciplinary Research Journal.

M. Pejic-Bach, T. Bertoncel, M. Meško, and Ž. Krsti?. 2020. Text mining of industry 4.0 job advertisements. Int J Inf Manage. 50: 416–431.

K. Fabian, E. Taylor-Smith, S. Smith, and A. Bratton. 2023. Signalling new opportunities? An analysis of UK job adverts for degree apprenticeships. Higher Education, Skills and Work-Based Learning. 13(2): 299–314.

N. Kamaleson, D. Chu, and F. E. B. Otero. 2021. Automatic Information Extraction from Electronic Documents Using Machine Learning. dalam: M. Bramer and R. Ellis (Eds). Artificial Intelligence XXXVIII. Cham: Springer International Publishing: 183–194.

H. Bao, L. Dong, W. Wang, N. Yang, S. Piao, and F. Wei. 2024. Fine-tuning pretrained transformer encoders for sequence-to-sequence learning. International Journal of Machine Learning and Cybernetics. 15(5): 1711–1728.

J.-C. Klie, R. E. de Castilho, and I. Gurevych. 2023. Analyzing Dataset Annotation Quality Management in the Wild. Computational Linguistics. 50: 817–866. Diunduh di https://api.semanticscholar.org/CorpusID:259937704 tanggal 25 Juni 2025.

J. Lin, X. Li, and G. Pekhimenko. 2020. Multi-node Bert-pretraining: Cost-efficient Approach. CoRR. abs/2008.00177. Diunduh di https://arxiv.org/abs/2008.00177 tanggal 25 Juni 2025.

A. Rula and J. D’Souza. 2023. Procedural Text Mining with Large Language Models. Prosiding The 12th Knowledge Capture Conference 2023. New York, NY, USA.

J. Cabessa, H. Hernault, and U. Mushtaq. 2024. In-Context Learning and Fine-Tuning GPT for Argument Mining. Diunduh di http://arxiv.org/abs/2406.06699 tanggal 25 Juni 2025.

Q. Dong et al. 2024. A Survey on In-context Learning. dalam: Y. Al-Onaizan, M. Bansal, and Y.-N. Chen (Eds). Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Miami, Florida, USA: Association for Computational Linguistics: 1107–1128.

J. Seo et al. 2022. Plain Template Insertion: Korean-Prompt-Based Engineering for Few-Shot Learners. IEEE Access. 10: 107587–107597.

C. Pornprasit and C. Tantithamthavorn. 2024. Fine-tuning and prompt engineering for large language models-based code review automation. Inf Softw Technol. 175: 107523.

J. Li, A. Sun, J. Han, and C. Li. 2023. A Survey on Deep Learning for Named Entity Recognition : Extended Abstract. Prosiding 2023 IEEE 39th International Conference on Data Engineering (ICDE).

A. V. Patil, H. Dand, and S. Kadam. 2024. Identifying specific details from text to populate databases and generate summaries using Named Entity Recognition. INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT (IJSREM). 8(5): 1–6.

M. Shao, A. Basit, R. Karri, and M. Shafique. 2024. Survey of Different Large Language Model Architectures: Trends, Benchmarks, and Challenges. IEEE Access. 12: 188664–188706.

C. Kauf et al. 2023. Event Knowledge in Large Language Models: The Gap Between the Impossible and the Unlikely. Cogn Sci. 47.

A. Vaswani et al. 2017. Attention Is All You Need. Diunduh di http://arxiv.org/abs/1706.03762 tanggal 25 Juni 2025.

Google T. 2025. Gemini: A Family of Highly Capable Multimodal Models. Diunduh di https://arxiv.org/abs/2312.11805 tanggal 25 Juni 2025.

Google T. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. Diunduh di https://arxiv.org/pdf/2403.05530 tanggal 25 Juni 2025.

L. Wang et al. 2024. Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs. NPJ Digit Med. 7(1): 41.

J. D. Rathod. 2024. Systematic Study of Prompt Engineering. Int J Res Appl Sci Eng Technol. 12(6): 597–613.

S. Sivarajkumar, M. Kelley, A. Samolyk-Mazzanti, S. Visweswaran, and Y. Wang. 2024. An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing: Algorithm Development and Validation Study. JMIR Med Inform. 12.

G. Shan. 2022. Monte Carlo cross-validation for a study with binary outcome and limited sample size. BMC Med Inform Decis Mak. 22(1).