In some classes when we turned in a project all we got was a grade, but when I got a low score I could usually figure out what was wrong eventually. Working as a proctor in a computer science class for non-majors, though, the students could use a bit more feedback. The problem was, it was tedious manually uploading 75 comments files for each project since the only method of uploading was a web interface.
When I started as a proctor, I didn’t expect this problem. Upload scripts from previous semesters had been passed down and looked promising… but failed.
A quick look over the script was enough to identify the problem. The web addresses in the script were http:// when what I really needed to use was https://. The obvious solution was to update the addresses in the script, which of course still failed.
My next idea was to use replace the whole curl part of the script and instead use wget. However, wget doesn’t support multipart form data so that was another dead end.
At this point, I figured I was stuck using the browser, but maybe a Greasemonkey script would be able to perform bulk uploads. After all, it’s just adding extra forms to a page and submitting them. But no, modern browsers don’t let you auto-fill the path of file input in forms, which in retrospect makes complete sense.
So I was stuck with curl, which I was already able to use to set cookies or to fetch a page or text file. However, downloading zip files also wasn’t working. Time to look up the curl man page and just try anything:
- Added
-ito also include the headers for extra feedback - Added
-referer, although no one in their right mind would rely on the referrer given - Added
-Ato specify Firefox as user agent, also dubious - Used
-H 'Accept-Encoding: gzip,deflate'to attempt zip downloads - Changed the keep-alive settings
Nothing helped.
I switched between using the -d and -F flags to specify the content type, but only got Error - bad file data and 403 Forbidden respectively. One of these must at least be a step in the right direction, but which was it?
Finally, I figured out how to use --trace-ascii dump.txt to see exactly what was going on but didn’t know what to look for until downloading an HTTP analyzer for Firefox and using that for comparison. At last things finally started making some sense. As noted in a blog post from Ian Dennis Miller, an Expect: 100-continue header was also in the trace.
I added -H 'Expect:' to the command and it finally happened. I uploaded a comment file remotely. Halfway through the semester and finally some down time to get things working. Damn, I love Spring Break.