Difference between revisions of "Summer Milestone 10 lucamatsumoto Macro Work Submission"

From crowdresearch
Jump to: navigation, search
 
(12 intermediate revisions by the same user not shown)
Line 34: Line 34:
 
:: [[File:SubmitWork.png]]
 
:: [[File:SubmitWork.png]]
  
 +
 +
:'''Option 3: File Grid'''
 +
 +
:If requester posts multiple files for the task and wanted to get a result file for each, we can use a grid style UI. The worker can see the progress too.
 +
: [[File:FileGrid.png]]
 +
 +
 +
:'''Option 4: Submit Work Page with Bulk Uploader'''
 +
 +
:The first screen has a list of uploaded files. If "Add..." button is pressed, the second window will pop up for bulk file uploading.
 +
: [[File:SubmitWork2.png]]  [[File:BulkFileUploader.png]]
 +
 +
<br/>
 +
=== Implementation Status ===
 +
 +
[8/12/2015]
 +
 +
I've been working on the front-end side of the page in Milestone 12.
 +
<br/>
 +
 +
 +
:'''Screenshot 1: Modal Uploader'''
 +
 +
: [[File:lucaSubmitWorkImpl1.png]]
 +
::This is about the same as Option 4 design. However, I faced some issues with this.
 +
::* This requires '''bootstrap.css'''. I am not sure if we can add it to the head for all. I tried to add it just for this page but couldn't make it work.
 +
::* I am using '''ngFileUpload''' for file uploading, and '''ngStrap''' for modal. Somehow, drag&drop or mouse click doesn't work on modal. I couldn't figure out why.
 +
<br/>
 +
 +
:'''Screenshot 2: Simplified Version'''
 +
 +
: [[File:lucaSubmitWorkImpl2.png]]
 +
::This is much simpler than Option 4. In stead of spending too much time on troubleshooting the Screenshot 1 design, I moved the file upload area to the first page.
 +
::* This doesn't require '''bootstrap.css'''.
 +
::* '''ngFileUpload''' seems to work.
 +
 +
<br/>
 +
:So for now, I will use the Screenshot 2 and make it fully functional. Then, will figure out the issue with ngFileUpload and ngStrap combination if needed.
 +
<br/>
 +
 +
[8/14/2015]
 +
 +
:'''Screenshot 3: Changed to MD style'''
 +
 +
: [[File:lucaSubmitWorkImpl4.png]]
 +
 +
====[8/16/2015]====
 +
 +
:'''Screenshot 4: Current Implementation'''
 +
 +
: [[File:lucaSubmitWorkImpl5.png]]
 +
 +
:'''What it does'''
 +
:* Uploads multiple files in any format
 +
:* Drag and Drop, or click dotted-area to select local files
 +
:* Deletes uploaded file and RequesterInputFile when red "delete" button on the screen is pressed
 +
:* When Cancel button is pressed, uploaded files will be deleted
 +
:* When Submit button is pressed, it returns a list of RequesterInputFile ids.
 +
 +
:'''Implementation Note'''
 +
:* Uses RequesterInputFile as Durim suggested. To upload non-csv files, some methods are added to CSVManagerViewSet and RequesterInputFileSerializer
 +
 +
:'''Changes from the last PR #367'''
 +
::'''As suggested by Durim'''
 +
::* Uploaded Files is md-list instead of st-table
 +
 +
::'''As suggested by Rohit'''
 +
::* macroWorkSubmitController is renamed to MacroWorkSubmitController
 +
::* Instead of custom "bytes", it uses existing "angular-filter"
 +
::* Existence check is added to "watch"
 +
::* Uses scss and moved styles to different files
 +
::* Uses "controller as" syntax
 +
 +
:'''Other Comments'''
 +
:* It is still using delete method on QuerySet to delete database record. I tried delete method on instance but it didn't delete database record.
 +
:* Couldn't figure out how to make the upload function reusable.
 +
:* Not sure if we really need "task info" and "comment" section when it is in the end-to-end flow. Will remove them if we don't need.
 +
 +
 +
====[8/20/2015]====
 +
 +
:'''Screenshot 5: Updated Screen Image'''
 +
 +
: [[File:lucaSubmitWorkImpl6.png]]
 +
 +
 +
====[8/26/2015]====
 +
 +
:'''Screenshot 6: Updated Screen Image'''
 +
 +
: [[File:lucaSubmitWorkImpl8.png]]
 +
 +
:* Made file list width shorter
 +
:* Revised list style based on the Material Design spec
 +
:* Added tooltip for delete button
 +
 +
:'''Implementation note on $watch for file upload'''
 +
:I didn't use the third parameter for deep comparison. The object this function is watching is an array, and ng-file-upload updates it. The "new value" from ng-file-upload contains the list of files being uploaded. Most likely, the files are different every time. So comparing old value and new value means comparing last uploaded file and currently uploading file. This is not a useful comparison. Instead, I compare the files already uploaded by that time, and currently uploading files to exclude duplicates. And furthermore, if I use “true” as the third parameter, it gives "TypeError" in the AngularJS itself ;(
 +
 +
 +
'''Next Step'''
 +
:* Make upload function reusable
 +
:* Integrate this to the end-to-end flow
 +
 +
<br/>
 
=== Open Questions ===
 
=== Open Questions ===
 
1. Google Drive/Drop Box Integration (nice to have feature?)
 
1. Google Drive/Drop Box Integration (nice to have feature?)
Line 49: Line 154:
 
=== Data Model ===
 
=== Data Model ===
  
−
Current TaskWorkResult model is like this.
+
<strike>Current TaskWorkResult model is like this.</strike> TaskWorkerResult is for template tasks. It may not be related to this.
  
 
  TaskWorkerResult
 
  TaskWorkerResult
Line 58: Line 163:
 
     status = IntegerField
 
     status = IntegerField
  
−
It has just one TextField for result. We need a way to associate multiple files to the TaskWorkResult.
+
It has just one TextField for result. We need a way to associate multiple files to the Task.
  
 
  Work
 
  Work
Line 66: Line 171:
 
     file_count    IntegerField
 
     file_count    IntegerField
 
      
 
      
−
The file count field above is to keep the number of files originally uploaded. This may be useful because the files may be deleted later or requesters my lose them after moving them to Google Drive/Drop Box.
+
The file count field above is to keep the number of files originally uploaded. This may be useful because the files may be deleted later.
  
−
  WorkFiles
+
  WorkFile
 
     id        AutoField
 
     id        AutoField
 
     work      ForeignKey (id) -> Work
 
     work      ForeignKey (id) -> Work
−
     link     TextField (or URLField)
+
     file     ForeignKey (id) -> RequesterInputFile
  
 
=== URL ===
 
=== URL ===

Latest revision as of 21:14, 26 August 2015

Macro Work Submission Page

This is the rough design idea of the page to submit works for macro tasks.

Design Considerations

Macro Work v.s. Micro Work
What is the difference between macro and micro work? Although they are not fundamentally different, I would imagine,
  • Since macro tasks take longer time than micro tasks, the result document may be a lot longer, and it may have a complex format. It will be in binary format like PDF, or application specific files such as doc, ppt, xls, mp3 etc.
  • Requesters may or may not know how many files workers will create. For example, if the task is programming in Java, the number of source codes and their names may vary.
  • Requesters may want to specify file name patterns in some cases.
  • How do requesters get the files submitted by workers? Download manually from our site, or should we provide a convenient way?
Features for Macro Work Submission
  • Upload binary file(s)
  • Use the file names given by workers
  • Requester can specify file naming convention
  • Requester Google Drive/Drop Box integration to get the files (nice to have?)

Page Designs

The following are the page design ideas. It doesn't include all the features listed above.


Option 1: Add "file upload form" to the prototype page
If we have "file upload form" in our template, we can create a prototype page like this.
PrototypeWithUpload.png


Option 2: Create a new page to upload multiple files
To submit many files at one time with comments, we can add a page like this.
SubmitWork.png


Option 3: File Grid
If requester posts multiple files for the task and wanted to get a result file for each, we can use a grid style UI. The worker can see the progress too.
FileGrid.png


Option 4: Submit Work Page with Bulk Uploader
The first screen has a list of uploaded files. If "Add..." button is pressed, the second window will pop up for bulk file uploading.
SubmitWork2.png BulkFileUploader.png


Implementation Status

[8/12/2015]

I've been working on the front-end side of the page in Milestone 12.


Screenshot 1: Modal Uploader
LucaSubmitWorkImpl1.png
This is about the same as Option 4 design. However, I faced some issues with this.
  • This requires bootstrap.css. I am not sure if we can add it to the head for all. I tried to add it just for this page but couldn't make it work.
  • I am using ngFileUpload for file uploading, and ngStrap for modal. Somehow, drag&drop or mouse click doesn't work on modal. I couldn't figure out why.


Screenshot 2: Simplified Version
LucaSubmitWorkImpl2.png
This is much simpler than Option 4. In stead of spending too much time on troubleshooting the Screenshot 1 design, I moved the file upload area to the first page.
  • This doesn't require bootstrap.css.
  • ngFileUpload seems to work.


So for now, I will use the Screenshot 2 and make it fully functional. Then, will figure out the issue with ngFileUpload and ngStrap combination if needed.


[8/14/2015]

Screenshot 3: Changed to MD style
LucaSubmitWorkImpl4.png

[8/16/2015]

Screenshot 4: Current Implementation
LucaSubmitWorkImpl5.png
What it does
  • Uploads multiple files in any format
  • Drag and Drop, or click dotted-area to select local files
  • Deletes uploaded file and RequesterInputFile when red "delete" button on the screen is pressed
  • When Cancel button is pressed, uploaded files will be deleted
  • When Submit button is pressed, it returns a list of RequesterInputFile ids.
Implementation Note
  • Uses RequesterInputFile as Durim suggested. To upload non-csv files, some methods are added to CSVManagerViewSet and RequesterInputFileSerializer
Changes from the last PR #367
As suggested by Durim
  • Uploaded Files is md-list instead of st-table
As suggested by Rohit
  • macroWorkSubmitController is renamed to MacroWorkSubmitController
  • Instead of custom "bytes", it uses existing "angular-filter"
  • Existence check is added to "watch"
  • Uses scss and moved styles to different files
  • Uses "controller as" syntax
Other Comments
  • It is still using delete method on QuerySet to delete database record. I tried delete method on instance but it didn't delete database record.
  • Couldn't figure out how to make the upload function reusable.
  • Not sure if we really need "task info" and "comment" section when it is in the end-to-end flow. Will remove them if we don't need.


[8/20/2015]

Screenshot 5: Updated Screen Image
LucaSubmitWorkImpl6.png


[8/26/2015]

Screenshot 6: Updated Screen Image
LucaSubmitWorkImpl8.png
  • Made file list width shorter
  • Revised list style based on the Material Design spec
  • Added tooltip for delete button
Implementation note on $watch for file upload
I didn't use the third parameter for deep comparison. The object this function is watching is an array, and ng-file-upload updates it. The "new value" from ng-file-upload contains the list of files being uploaded. Most likely, the files are different every time. So comparing old value and new value means comparing last uploaded file and currently uploading file. This is not a useful comparison. Instead, I compare the files already uploaded by that time, and currently uploading files to exclude duplicates. And furthermore, if I use “true” as the third parameter, it gives "TypeError" in the AngularJS itself ;(


Next Step

  • Make upload function reusable
  • Integrate this to the end-to-end flow


Open Questions

1. Google Drive/Drop Box Integration (nice to have feature?)

Requesters may want to receive the work files in their Google Drive/Drop Box folder. If so, the files should not be uploaded to Google Drive/Drop Box directly. It should be done through Daemo. If we allow workers to upload files directly to Google Drive/Drop Box, we can't keep track of the work submission.

2. How long should we keep the files?

If we store uploaded files in our database, it will eat up the storage space. Should we ask requesters to download the files and remove them from our storage after a while?

END


Below is just for my memo. I need to think more on these.

Data Model

Current TaskWorkResult model is like this. TaskWorkerResult is for template tasks. It may not be related to this. 

TaskWorkerResult
    task_worker = ForeignKey
    result = TextField
    template_item = ForeignKey
    statuses = ((1, 'Created'),(2, 'Accepted'),(3, 'Rejected'))
    status = IntegerField

It has just one TextField for result. We need a way to associate multiple files to the Task. 

Work
    id             AutoField
    task           ForeignKey (id) -> Task
    comment        TextField
    file_count     IntegerField

The file count field above is to keep the number of files originally uploaded. This may be useful because the files may be deleted later. 

WorkFile
    id        AutoField
    work      ForeignKey (id) -> Work
    file      ForeignKey (id) -> RequesterInputFile

URL

Page URL: /work/
API URL:  /api/work/
Retrieved from "http://crowdresearch.stanford.edu/w/index.php?title=Summer_Milestone_10_lucamatsumoto_Macro_Work_Submission&oldid=11820"