Tying of embeddings for improving regularization in neural networks for named entity recognition task
Тип публікації :
Стаття
Дата випуску :
2018
Автор(и) :
Bevza, M. V.
Мова основного тексту :
Англійська
eKNUTSHIR URL :
Випуск :
3
ISSN :
1812-5409
Початкова сторінка :
59
Кінцева сторінка :
64
Цитування :
[APA 7] Bevza, M. V. (2018). Tying of embeddings for improving regularization in neural networks for named entity recognition task. Bulletin of Taras Shevchenko National University of Kyiv. Physics and Mathematics, (3), 59–64. https://doi.org/10.17721/1812-5409.2018/3.8
[ДСТУ] Bevza M. V. Tying of embeddings for improving regularization in neural networks for named entity recognition task. Bulletin of Taras Shevchenko National University of Kyiv. Physics and Mathematics. 2018. no. 3. P. 59—64. DOI: 10.17721/1812-5409.2018/3.8 (date of access: 25.07.2026).
We analyze neural network architectures that yield state of the art results on named entity recognition task and propose a new architecture for improving results even further. We have analyzed a number of ideas and approaches that researchers have used to achieve state of the art results in a variety of NLP tasks. In this work, we present a few of them which we consider to be most likely to improve existing state of the art solutions for named entity recognition task. The architecture is inspired by recent developments in language modeling task. The suggested solution is based on a multi-task learning approach. We incorporate part of speech tags as input for the network. Part of speech tags to be yielded by some state of the art tagger and also ask the network to produce those tags in addition to the main named entity recognition tags. This way knowledge distillation from a strong part of speech tagger to our smaller network is happening. We hypothesize that designing neural network architecture in this way improves the generalizability of the system and provide arguments to support this statement.Key words: neural networks, named entity recognition, regularization, generalizability, multi-task learning.Pages of the article in the issue: 59 - 64Language of the article: Ukrainian
Файл(и) :![Ескіз]()
Вантажиться...
Формат :
Adobe PDF
Розмір :
872.86 KB
Контрольна сума :
(MD5):044401a7f4a007157fa1cc7afe2b0eff
Якщо не вказано інше, ця робота розповсюджується на умовах ліцензії Creative Commons Attribution 4.0 International