Abstract:
Effective communication between teachers and students is essential in today’s educational
environment. This project presents a transformer model that has been designed for the ed-
ucational environment. With the use of this model, students can have personalized learning
experiences and ask academic questions and while getting immediate answers generated by
the model. The personalized feedback is given by using a deep learning model. Due to lack
of dataset existing in education domain, a dataset on education domain is developed using
web scraping techniques, which scrapes or extracts the relevant data from internet. The data
which is obtained through web scrapping is given to the model for training. The deep learn-
ing model used is Generative Pre-trained Transformer-2 (GPT-2) a transformer model which
then analyses the pattern and gives the customized feedback back to student. The GPT-2
was used for training on the data set. After performing the validation between reference
and generated text, the evaluation metrics were obtained. The evaluation metrics showed
very low BLEU score and ROUGE score. To increase the value of BLEU score and ROUGE
score, data augmentation was performed. Data augmentation technique such as synonym
replacement was performed to increase the data size. Parameter efficient fine-tuning tech-
niques such as Low rank Adaption is applied to reduce the size of the model which in turn
can produce the results of the model at much less computation time and improved memory
efficiency. The integration of this model is expected to transform the educational system
by creating an atmosphere in which students are inspired to learn and teachers can skilfully
meet each student’s unique learning needs.
