Difference between revisions of "Annotatrix/Work plan"

From Apertium
Jump to navigation Jump to search
 
(52 intermediate revisions by one other user not shown)
Line 1: Line 1:
== Workflow ==
== Workflow ==
=== Week 3 Django warm-up ===
=== Week 14 TSX_manger and trainer application hand-in===
From 28th to 4th of June
''from 13th to 18th of August''

*Implement the first interface (Insert file view)
Trainer and tsx_manager hand-in
**Managing files
*Documented trainer from developer side '''Done'''
**Upload files
*Documented tsx_manager from developer side '''Done'''
**Managing users authentication
*Documentation for final users
**Getting history of user's trained

Tagger improvements
*Able to download corpus text files from training details view '''Done'''
*Fixed use metadix extension instead of just dix to tag a corpus '''Done'''
*Added button to download corpus tagged from index and training_detail views '''Done'''

Third application (On_fly Trainer)
*Tagger>Trainer able to modify the words per page '''Done'''
*Added trainer to the main index '''Done'''
*No untagged corpus file need more for new trainer on_fly '''Done'''
*Trainer on_fly working and training corpora '''Done'''
*Added insert new trainer to trainer app '''Done'''
*Added trainer app with basics and controller code '''Done'''

Second application (tsx_manager)
*Download tsx file option '''Done'''
*Able to modify and refresh the tsx file '''Done'''
*Fixed Multi-labels tags does not appearing '''Done'''
*Fixed insert+modified acctions '''Done'''
*Tsx manager ready for to document it '''Done'''
*Tested labels acctions from view '''Done'''
*Tested enforce and preferences acctions from view '''Done'''
*Tested mult and fobid acctions from view '''Done'''
*Able to modify label and mult labels '''Done'''
*Able to modify labels, enforce and forbid rules '''Done'''
*refactorized insertions code and fixed a minnor error '''Done'''
*Able to insert new categories and download the new tsx file version modified '''Done'''

=== Week 13 TSX_manger application hand-in===
''from 6th to 12th of August''

Second application (tsx_manager)
*Global search option '''Done'''
*Upload tsx file
**Persistence files to manage them and have history from them '''Done'''
**View to upload the tsx file '''Done'''
**Connect insert to details tsx files '''Done'''

*Edition of tsx_fields
**def_label
***Fixed index for labels '''Done'''
***Modify '''Done'''
***insert '''Done'''
***remove '''Done'''
**def_mult_label
***Modify correctly '''Done'''
***insert correctly '''Done'''
**forbid
***Fixed index for forbids '''Done'''
***Modify '''Done'''
***insert '''Done'''
***remove '''Done'''


=== Week 12 TSX_manger application===
''from 30th to 5th of August''

Second application (tsx_manager)
*Edition of tsx_fields
**def_mult
***New insert form for mult_rules '''Done'''
***Modify mult_rules '''Done'''
***Insert enforce rules '''Done'''

(Final exam on University) '''Done'''

=== Week 11 TSX_manger application ===
''from 23th to 29th of August''

Second application (tsx_manager)
*Edition of tsx_fields
**def_mult_label
***Remove enforce rules '''Done'''
***Insert enforce rules '''Almost Done'''
**enforce
***Modify enforce rules '''Done'''
***Insert enforce rules '''Done'''
***Remove enforce rules '''Done'''
**preferences
***Update prefers '''Done'''
***Insert prefers '''Done'''
***Remove prefers '''Done'''

=== Week 10 Trainer view deliverable, and start next application ===
''From 15th to 22nd of August''

*Second application (tsx_manager)
**Study tsx files format '''Done'''
**Design python objects to manage tsx files from python '''Done'''
**Implement function to transform tsx files to python objects '''Done'''
**Start tsx_manager in Django '''Done'''
**Implement the first view empty (url, views, models, etc) in Django '''Done'''
**Show the information from Django to tsx_manager view '''Done'''
**Apply JavaScript plug-in to filter each category '''Done'''
**Show information using tabs '''Done'''

*Second deliverable hand-in
**Documentation for final users installation '''Done'''
**Documentation for final users how to use the system '''Done'''
***The second deliverable is places [http://wiki.apertium.org/wiki/Getting_started_with_Annotatrix#How_to_use_Annotatrix second deliverable]

=== Week 9 Trainer view, JavaScript and backend improvements ===
''From 8th to 14th of August''

*Improve Trainer use to suit with Gema idea
**Partial saved state of the training '''Done'''
**Right panel and corpus info '''Done'''
**Get keyboard keys from JavaScript '''Done'''
**Get use to with jQuery '''Done'''
**Key recorder for tag alternative selection '''Done'''
***Now the trainer is able to train corpora using just the keyboard, with a friendly user interface, cleaned corpus to disambiguate and it is paginated

*Second deliverable hand-in
**Documentation for developers '''Done'''

=== Week 8 Trainer view, JavaScript and backend improvements ===
''From 1st to 7th of August''

*Let users upload .dix and .tsx files to train the corpus '''Done'''
*Train corpus with user own dix and tsx files '''Done'''

*Improve Trainer use to suit with Gema idea
**Paginator '''Done'''

*Frontend end users
**Study CSS '''Done'''
**Study JavaScript '''Done'''
**Study JQuery '''Done'''
**Study Ajax '''Some part Deprecated'''
**Install Bootstrap '''Done'''
**Insert navbar '''Done'''
**Button links to go to index '''Done'''
***Updated all the templates with new similar layout '''Done'''
***Improved the style of the system using Bootstrap '''Done'''
**Improved the server time and less petitions using Javascript '''Done'''

*Managing users authentication
**Django-registration package '''Done'''
***Autentiaction using email confirmation and just active users can see the system '''Done'''

=== Week 7 Trainer view and JavaScript ===
''From 25th to 31th of July''
*Implement the second interface (Trainer view)
**Backend test to train corpus '''Done'''
**Implement backend to train corpus '''Done'''
**Implement first version of Frontend to train corpus '''Done'''
**Apply first deliverable feedback '''Done'''
**Improve trainer save status to reduce file size '''Done'''
***''Using es.tagged.txt reduced from 2.6Mb to 16Kb!, more than a 99% of improvement''

*Frontend end users
**Study CSS '''Some part'''
**Study JavaScript '''Some part'''
***Apply to the inser_corpus view '''Done'''
**Study Ajax '''Some part'''
**Try to use directly plain HTML to train the corpus using combo boxes '''Done'''

=== Week 6 Trainer view ===
From 18th to 24th of July
*Implement the second interface (Training view)
**More TDD tests for Training view '''Done'''
**Made up again the training model '''Done'''
**Made up a test to test the prob generated '''Done'''
**Implement the train method for a tagged_corpus method to generate the prob file with the path as argument '''Done'''
**Change all the path to relatives repositories '''Done'''
**Show the tagged_corpus in a correct textarea '''Done'''

=== Week 5 First deliverable and Training ===
From 12th to 18th of July
*Implement the second interface (Training view)
**TDD tests for Training view '''Done'''
**Made up again the training model '''Important part Done'''

*Document the first view '''Done'''
**Document for developers '''Done'''
**Document for users '''Done'''
'''You can find the first deliverable [http://wiki.apertium.org/wiki/Getting_started_with_Annotatrix here]'''

*Implement the system's modules '''Done'''
**Implement controller to use apertium-dbus '''Done'''
**Implement external models for Tags '''Done'''

=== Week 4 Insert corpus view ===
From 5th to 11th of July

*Implement a index annonatrix view to access to the different apps '''Done'''
*Implement all links to see details and insert corpora '''Done'''

*Testing the first view (by TestCases) '''Done'''
**Learn how to test with TestCases in Django '''Done'''
**Implement test for the first view '''Done'''

*Implement the first view (Insert file view) '''Done'''
**First basics views '''Done'''
**Managing files '''Done'''
**Upload files '''Done'''
**Select corpus language '''Done'''
***Upload Corpus model with language field '''Done'''
**Add title field to the view '''Done'''
**Go to tagger button '''Done'''
***[https://docs.djangoproject.com/en/1.5/topics/forms/modelforms/ modelforms] '''Done'''
***[https://docs.djangoproject.com/en/1.5/topics/forms/formsets/#understanding-the-managementform understanding-the-managementform] '''Done'''
**Validate "go to tagger" forms '''Done'''

*Friendly interface
**HTML tutorial w3schools.com '''Done'''
**CSS Tutorial w3schools.com '''Done'''

*Implement the system's modules
*Implement the system's modules
**Implement controller to use apertium-dbus
**Implement controller to use apertium-dbus '''Some Part'''
**Implement external models for Tags
**Implement external models for Tags
**Implement internal models for trainings
**Implement internal models for trainings '''Done'''
**Implement Django administration site '''Done'''
*Design a system directly to web

**Design controller to use apertium-dbus
=== Week 3 Django warm-up ===
**Design external models for Tags
From 28th of June to 4th of July
*Design a system directly to web '''Done'''
**Design controller to use apertium-dbus '''Done'''
**Design external models for Tags '''Done'''
**Design internal models for trainings '''Done'''
**Design internal models for trainings '''Done'''
**Design structure for annotatrix project '''Done'''
**Design structure for annotatrix project '''Done'''
Line 43: Line 255:


== System design ==
== System design ==

[[File:UML_global_v2.jpg|500px]]


There are just one Django project but 4 different applications
There are just one Django project but 4 different applications
Line 53: Line 263:


=== Internal Django models package ===
=== Internal Django models package ===
'''User'''

Default user Django model to use authentication and also save
the password

'''Corpus'''
'''Corpus'''

vars:
*title
*corpus-file


'''Training'''
'''Training'''

vars:
*corpus-tagged
*prob-file
*finalized
*date
*corpus


=== External Controller package ===
=== External Controller package ===
'''tagger-ctrl'''
'''tagger-ctrl'''

methods:
*File tag_by_path(path_file)
*File tag_by_string(corpus_in_string_to_tag)
*[string] get_pairs() """it returns a list of pair of languages already installed in the system"""
*path_to_prob_file train(path_to_corpus, pair_direction)

=== External Models package ===
'''tag'''

methods:
*select_alt(num)
vars:
*string lemma
*[string] alternatives
*integer selected


== Working plan ==
== Working plan ==
Line 139: Line 317:
|-
|-
|}
|}

==Todo==


=== Coding ===

* Structure the repository
* Set up a Makefile
* <s>Add an AUTHORS file</s> -- r44881
* <s>Make loaders for the <code>.ui</code> files so people see them working.</s> -- r44915


=== to discuss ===

* Use Python2.7 and gtk2.24 or use Python3
* How to manage big data files (wikipedia dumps), we could use a temporal file but maybe this file could be too big in memory consumption terms


==External links==
==External links==

Latest revision as of 15:47, 10 October 2013

Workflow[edit]

Week 14 TSX_manger and trainer application hand-in[edit]

from 13th to 18th of August

Trainer and tsx_manager hand-in

  • Documented trainer from developer side Done
  • Documented tsx_manager from developer side Done
  • Documentation for final users

Tagger improvements

  • Able to download corpus text files from training details view Done
  • Fixed use metadix extension instead of just dix to tag a corpus Done
  • Added button to download corpus tagged from index and training_detail views Done

Third application (On_fly Trainer)

  • Tagger>Trainer able to modify the words per page Done
  • Added trainer to the main index Done
  • No untagged corpus file need more for new trainer on_fly Done
  • Trainer on_fly working and training corpora Done
  • Added insert new trainer to trainer app Done
  • Added trainer app with basics and controller code Done

Second application (tsx_manager)

  • Download tsx file option Done
  • Able to modify and refresh the tsx file Done
  • Fixed Multi-labels tags does not appearing Done
  • Fixed insert+modified acctions Done
  • Tsx manager ready for to document it Done
  • Tested labels acctions from view Done
  • Tested enforce and preferences acctions from view Done
  • Tested mult and fobid acctions from view Done
  • Able to modify label and mult labels Done
  • Able to modify labels, enforce and forbid rules Done
  • refactorized insertions code and fixed a minnor error Done
  • Able to insert new categories and download the new tsx file version modified Done

Week 13 TSX_manger application hand-in[edit]

from 6th to 12th of August

Second application (tsx_manager)

  • Global search option Done
  • Upload tsx file
    • Persistence files to manage them and have history from them Done
    • View to upload the tsx file Done
    • Connect insert to details tsx files Done
  • Edition of tsx_fields
    • def_label
      • Fixed index for labels Done
      • Modify Done
      • insert Done
      • remove Done
    • def_mult_label
      • Modify correctly Done
      • insert correctly Done
    • forbid
      • Fixed index for forbids Done
      • Modify Done
      • insert Done
      • remove Done


Week 12 TSX_manger application[edit]

from 30th to 5th of August

Second application (tsx_manager)

  • Edition of tsx_fields
    • def_mult
      • New insert form for mult_rules Done
      • Modify mult_rules Done
      • Insert enforce rules Done

(Final exam on University) Done

Week 11 TSX_manger application[edit]

from 23th to 29th of August

Second application (tsx_manager)

  • Edition of tsx_fields
    • def_mult_label
      • Remove enforce rules Done
      • Insert enforce rules Almost Done
    • enforce
      • Modify enforce rules Done
      • Insert enforce rules Done
      • Remove enforce rules Done
    • preferences
      • Update prefers Done
      • Insert prefers Done
      • Remove prefers Done

Week 10 Trainer view deliverable, and start next application[edit]

From 15th to 22nd of August

  • Second application (tsx_manager)
    • Study tsx files format Done
    • Design python objects to manage tsx files from python Done
    • Implement function to transform tsx files to python objects Done
    • Start tsx_manager in Django Done
    • Implement the first view empty (url, views, models, etc) in Django Done
    • Show the information from Django to tsx_manager view Done
    • Apply JavaScript plug-in to filter each category Done
    • Show information using tabs Done
  • Second deliverable hand-in
    • Documentation for final users installation Done
    • Documentation for final users how to use the system Done

Week 9 Trainer view, JavaScript and backend improvements[edit]

From 8th to 14th of August

  • Improve Trainer use to suit with Gema idea
    • Partial saved state of the training Done
    • Right panel and corpus info Done
    • Get keyboard keys from JavaScript Done
    • Get use to with jQuery Done
    • Key recorder for tag alternative selection Done
      • Now the trainer is able to train corpora using just the keyboard, with a friendly user interface, cleaned corpus to disambiguate and it is paginated
  • Second deliverable hand-in
    • Documentation for developers Done

Week 8 Trainer view, JavaScript and backend improvements[edit]

From 1st to 7th of August

  • Let users upload .dix and .tsx files to train the corpus Done
  • Train corpus with user own dix and tsx files Done
  • Improve Trainer use to suit with Gema idea
    • Paginator Done
  • Frontend end users
    • Study CSS Done
    • Study JavaScript Done
    • Study JQuery Done
    • Study Ajax Some part Deprecated
    • Install Bootstrap Done
    • Insert navbar Done
    • Button links to go to index Done
      • Updated all the templates with new similar layout Done
      • Improved the style of the system using Bootstrap Done
    • Improved the server time and less petitions using Javascript Done
  • Managing users authentication
    • Django-registration package Done
      • Autentiaction using email confirmation and just active users can see the system Done

Week 7 Trainer view and JavaScript[edit]

From 25th to 31th of July

  • Implement the second interface (Trainer view)
    • Backend test to train corpus Done
    • Implement backend to train corpus Done
    • Implement first version of Frontend to train corpus Done
    • Apply first deliverable feedback Done
    • Improve trainer save status to reduce file size Done
      • Using es.tagged.txt reduced from 2.6Mb to 16Kb!, more than a 99% of improvement
  • Frontend end users
    • Study CSS Some part
    • Study JavaScript Some part
      • Apply to the inser_corpus view Done
    • Study Ajax Some part
    • Try to use directly plain HTML to train the corpus using combo boxes Done

Week 6 Trainer view[edit]

From 18th to 24th of July

  • Implement the second interface (Training view)
    • More TDD tests for Training view Done
    • Made up again the training model Done
    • Made up a test to test the prob generated Done
    • Implement the train method for a tagged_corpus method to generate the prob file with the path as argument Done
    • Change all the path to relatives repositories Done
    • Show the tagged_corpus in a correct textarea Done

Week 5 First deliverable and Training[edit]

From 12th to 18th of July

  • Implement the second interface (Training view)
    • TDD tests for Training view Done
    • Made up again the training model Important part Done
  • Document the first view Done
    • Document for developers Done
    • Document for users Done

You can find the first deliverable here

  • Implement the system's modules Done
    • Implement controller to use apertium-dbus Done
    • Implement external models for Tags Done

Week 4 Insert corpus view[edit]

From 5th to 11th of July

  • Implement a index annonatrix view to access to the different apps Done
  • Implement all links to see details and insert corpora Done
  • Testing the first view (by TestCases) Done
    • Learn how to test with TestCases in Django Done
    • Implement test for the first view Done
  • Implement the first view (Insert file view) Done
    • First basics views Done
    • Managing files Done
    • Upload files Done
    • Select corpus language Done
      • Upload Corpus model with language field Done
    • Add title field to the view Done
    • Go to tagger button Done
    • Validate "go to tagger" forms Done
  • Friendly interface
    • HTML tutorial w3schools.com Done
    • CSS Tutorial w3schools.com Done
  • Implement the system's modules
    • Implement controller to use apertium-dbus Some Part
    • Implement external models for Tags
    • Implement internal models for trainings Done
    • Implement Django administration site Done

Week 3 Django warm-up[edit]

From 28th of June to 4th of July

  • Design a system directly to web Done
    • Design controller to use apertium-dbus Done
    • Design external models for Tags Done
    • Design internal models for trainings Done
    • Design structure for annotatrix project Done
      • annotatrix as project and 4 different applications (tagger, GCManager, TSX-managaer and Prob measurer)
  • Do toy web app using Django Done
    • Django official tutorial Part 1 Done
    • Django official tutorial Part 2 Done
    • Django official tutorial Part 3 Done
    • Django official tutorial Part 4 Done
    • Django official tutorial Part 5 Done
    • Django official tutorial Part 6 Done
  • Advanced tutorial: How to write reusable apps Done

Week 2 Kick-off[edit]

From 27th to 27th of June

  • Resolve python2 or python3 discussion Python3
    • Learn differences python3 from python2 Done
    • Learn differences gtk2 and gtk3 using python3 Deprecated
    • Remake up the mock-ups UI using gtk3 Deprecated
    • Port launcher in python2 to python3 Deprecated
  • Made up a system design approaching implement a web app or a desktop one Updated
    • Design a system directly to web
  • Install Django and python3 Done

Week 1 Pre-Kick-off[edit]

  • Know how to use launch lt-proc to tag corpus Done
  • Install apertium-tolk and check the code Done
  • Finish the exams Done

System design[edit]

There are just one Django project but 4 different applications

There are 2 kind of packages, the Django ones and the external ones The Django packages are provided by Django as views, models, urls, etc The external packages are extra for this projects

Internal Django models package[edit]

Corpus

Training

External Controller package[edit]

tagger-ctrl

Working plan[edit]

Week Dates Tasks
0 21/05—16/06 Bonding and designing firsts internals GUIs
1 17/06—23/06 I'm still with final exams
2 24/06—30/06 Input File UI coding 30h, Tagger Corpus UI coding 5/60h
3 01/07—07/07 Tagger Corpus UI coding 35/60h
Deliverable 1
4 08/07—14/07 Tagger Corpus UI coding 20/60h + TDD 15/40h
5 15/07—21/07 Tagger Corpus UI TDD 25/40h, TSX file manager UI coding 10/40h
Deliverable 2
6 22/07—28/07 TSX file manager UI coding 30/40h + TDD 5/20h
7 29/07—04/08 TSX file manager UI TDD 15/20h, Constraint Grammar UI coding 20/50h
Deliverable 3 and Mid-term evaluations deadline August 2sd
8 05/08—11/08 Constraint Grammar UI coding 30/50h + TDD 5/30
9 12/08—18/08 Constraint Grammar UI TDD 25/30h, Performance measure .prob UI coding 10/70h
Deliverable 4
10 19/08—25/08 Performance measure .prob UI coding 35/70h
11 26/08—01/09 Performance measure .prob UI coding 25/70h + TDD 10/40h
12 02/09—08/09 Performance measure .prob UI TDD 30/40h, Final documentation 5h
Deliverable 5
13 09/09—15/09 Final documentation and finalitation
Final evaluation deadline September 27th

External links[edit]

Retrieved from "https://wiki.apertium.org/w/index.php?title=Annotatrix/Work_plan&oldid=44247"