Comprehensive API for Kurdish Text Processing Project

The project aims to build a comprehensive Application Programming Interface (API) for text processing written in Kurdish language (Arabic alphabet). The objectives of the project are: provide the capability to do file reading/writing, text segmentation, statistical analysis of text and perform information retrieval on texts written in Kurdish language. The project also aims to develop applications such as Kurdish spell checker, Kurdish thesaurus, and building domain ontologies for web services. If you interested in participating or sponsoring the project please contact me at armyunis@scs.carleton.ca

Title and Detailed description of individual Projects

1. Kurdish Word Bank (project description, coming soon)
2. Kurdish Spell Checker ( project description)
3. Kurdish Thesauri (project description, coming soon)
4. Common Words among Kurdish Dialects (project description, coming soon)
5. Most Used Kurdish Word (project description, coming soon)

Well-Knowing Stemming Alorithms

• Paice/Husk Stemming Algorithm,   Another Stemmer , Lancaster page
• Porter Stemming Algorithm,   An algorithm for suffix stripping , Porter page
• Lovins Stemming Algorithm,   Development of a Stemming Algorithm , Lovins page
• Dawson Stemming Algorithm,   Suffix removal and word conflation
• Krovetz Stemming Algorithm,   Viewing Morphology as an Inference Process

• Presentations

• Code (please contact me to obtain the most current code for processing Kurdish Text)

• References

• Participant

Coming soon

• Publications

Coming soon

• Resources and case studies

We have used the text book that has been used for teaching the first year elementary school students in Kurdistan to show some of the current Kurdish API futures. To download the book please clicks on the book cover.

Total number of unique words used in the text book is: 994. To see the list of all unique words used in the book click here .


Figure 1 shows the top 20 words used in the book. To see the frequency of each word used in the book click here


Figure 1


Table 1 presents the number of words starting with each letter. To see the words click here.



Table 1

ز ٨٢	ڕ ٥٤	ر ٠	د ٣٥٣	خ ٤٢	ح ٨	چ ٤٣	ج ٣٦	ت ٦٢	پ ٥٤	ب ٣١٨	ئ‍ ٢١٢
ڵ ٢	ل ٧٤	گ ٩٥	ك ٩٥	ق ٢٩	ڤ ٣	ف ٦	غ ٧	ع ١٠	ش ٢٩	س ٩٥	ژ ٣
		ێ ٠	ی ٣٥	ڵا ٠	لا ٠	ۆ ٠	وو ١٨	و ١١٢	ه‍ ٦٦	ن ١١٥	م ١٢٣


Table 2 is similar to the Table 1 except repeated words removed. To see the words remaining words click here.

Table 2

ز ٢٤	ڕ ٣٥	ر ٠	د ١٥٨	خ ٣١	ح ٤	چ ٣٣	ج ٢٢	ت ٢٩	پ ٣١	ب ١٤٧	ئ‍ ٦١
ڵ ٢	ل ٣٣	گ ٤٠	ك ٥٨	ق ١٧	ڤ ٣	ف ٥	غ ٥	ع ٤	ش ١٦	س ٤٩	ژ ٣
		ێ ٠	ی ٩	ڵا ٠	لا ٠	ۆ ٠	وو ١٠	و ١٧	ه‍ ٣٥	ن ٦١	م ٥١




• Relevant conferences and journals

CICLing Conference on Intelligent Text Processing and Computational Linguistics
Annual Meeting of the Association for Computational Linguistics
The International Conference on Asian Language Processing (IALP)
Conference on Computational Natural Language Learning
Workshop on Treebanks and Linguistic Theories



The table of Kurdish scripts in cursive format(different forms for the same letter )

General Unicode	Isolated	End	Middle	Beginning	General Unicode	Isolated	End	Middle	Beginning	General Unicode	Isolated	End	Middle	Beginning
u0626	ا‎	ﺎ‎	ﺎ‎	ئ‍‎	u0628	ب‎	ـب‎	ـبـ‎	بـ‎	u067e	پ‎	ـپ‎	ـپـ‎	پـ‎
u062A	ت‎	ـت‎	ـتـ‎	تـ‎	u062C	ج‎	ـج‎	ـجـ‎	جـ‎	u0686	ﭺ‎	ﭻ‎	ﭽ‎	ﭼ‎
u062D	ح‎	ـح‎	ـحـ‎	حـ‎	u062E	خ‎	ـخ‎	ـخـ‎	خـ‎	u062F	ﺩ‎	ـد‎	ـد‎	ﺩ‎
u0631	ر‎	ـر‎	ـر‎	ڕ‎	u0695	ڕ‎	ـڕ‎	ـڕ‎	ڕ‎	u0632	ز‎	ـز‎	ـز‎	ز‎
u0698	ژ‎	ـژ‎	ـژ‎	ژ‎	u0633	س‎	ـس‎	ـسـ‎	سـ‎	u0634	ش‎	ـش‎	ـشـ‎	شـ‎
u0639	ع‎	ـع‎	ـعـ‎	عـ‎	u063A	غ‎	ـغ‎	ـغـ‎	غـ‎	u0641	ف‎	ـف‎	ـفـ‎	فـ‎
uFB6A	ڤ‎	ـڤ‎	ـڤـ‎	ڤـ‎	u0642	ق‎	ـق‎	ـقـ‎	قـ‎	u0643	ك‎	ـك‎	ـكـ‎	كـ‎
u06AF	ﮒ‎	ﮓ‎	ﮕ‎	ﮔ‎	u0644	ل‎	ـل‎	ـلـ‎	لـ‎	u06B5	ڵ‎	ـڵ‎	ـڵـ‎	ڵـ‎
u0645	م‎	ـم‎	ـمـ‎	مـ‎	u0646	ن‎	ـن‎	ـنـ‎	نـ‎	u0647	ﻩ‎	ـه‎	ـهـ‎	هـ‎
u0648	و‎	ـو‎	ـو‎	و‎	u0648 u0648	وو‎	ـوو‎	ـوو‎	وو‎	u06C6	ۆ‎	ـۆ‎	ـۆ‎	ۆ‎
u0644 u0627	ﻻ‎	ﻼ‎	ﻼ‎	ﻻ‎	u06b5 u0627	ڵا‎	ـڵا‎	ـڵا‎	ڵا‎	uu06CC	ى‎	ـى‎	ـيـ‎	يـ‎
u06CE	ێ‎	ـێ‎	ـێـ‎	ێ‎	u0200C	u0200D	u0640



Unicode Character 'ZERO WIDTH NON-JOINER' (U+200C) for html ‌ or ‌ used in situations like this: ترێ‌ی, نوێ‌ی
Unicode Character 'ZERO WIDTH JOINER' (U+200D) for html ‍ or ‍ can be used to show the cursive forms of the letters, example   ـه   ـح   ـج
Unicode Character 'ARABIC TATWEEL' (U+0640) for html ـ used to extend the shape of a letter like this: کــــوردســــتـــان