* Download managers

[jmurphy]

I want to be able to capture the download history of the files I am collecting from different sites like Find My Past.

I use Firefox. If I "show all downloads" I can look at the history and it shows me the filenames and the site they came from, plus a timestamp.

It used to be that one could scrape the download history from the file where Firefox stores this, but they changed the place and manner of storing the history, so the old hacks no longer work.

My experience is that the user needs to flush the download history from time to time, or bad things happen, like Firefox getting boggy and stalling out while doing certain tasks.

Does anyone know how I can pipe the contents of the download history to a new file, before I clear the list and start over again with a clean plate? And can anyone recommend a download manager or extension that I could use to capture the website accession information?

The stamp on the filename gives me the day and time I captured the file, but not where it came from. I try to save these in named folders, but since Firefox dumps everything into the download directory, I don't always remember to move things to their named folders after my download session is over.

[tatewise]

Since you say the file date-stamps are satisfactory, and it is only the download folder that is a problem, then just change the Firefox download setting.

In Tools > Options, on the General tab, tick Always ask me where to save files.
That way you can always download to the appropriate folder.

If my recollection is correct Firefox remembers the last used download folder, so in a session with say FindMyPast it will offer the desired folder each time after the first download.

[jmurphy]

Well, yes, Mike, that would fix the problem until I downloaded something else and forgot to change the folder, not unlike forgetting to turn off Auto-Source-Citation in Family Historian.

Ideally I'd like a log of all the filenames downloaded and where they came from. I'd like something that would generate that, plus I was hoping I could find some way to scrape the information for the files which are already downloaded.

[tatewise]

It is not anywhere near as bad as FH Auto-Source-Citation, because Firefox puts up a dialogue with the download folder clearly displayed, and you have to confirm, or browse to a different folder.

[DavidNewton]

Ideally I'd like a log of all the filenames downloaded and where they came from. I'd like something that would generate that, plus I was hoping I could find some way to scrape the information for the files which are already downloaded.

This information is contained in a database file places.sqlite tucked away in your firefox profile folder. I have examined this file using SQLiteSpy and the information regarding downloads is in a table labelled moz_annos linked to another table labelled moz_places which contains urls website titles etc. Since I have zero knowledge of using SQL queries I haven't a clue how you might extract the information that you want.

David

[Jane]

I have not tried it, but a quick google showed up

http://www.historyviewer.net/index.html

[jmurphy]

I just downloaded HistoryViewer and gave it a try.

It's possible that I didn't let it sit and 'think' long enough, but it appears to work for all of the Firefox functions *except* the Download History.

I can't remember exactly when Mozilla changed the location of the download history, but I suspect that either I didn't wait long enough for the results to come up, or it's looking for the old file and not the current one.

Also, a warning: if you want it for its other features, the installer offers you the opportunity to install half-a-dozen other things at the same time, so be prepared to hit 'skip'.

It's too bad, because something simple like that would have worked nicely, if I could just dump the results out to a file or scrape them. I don't want anything fancy. Just a log I could use to double-check which multimedia items have been entered into FH, sources created, etc.

[tatewise]

I think there may a solution as follows:
Download & install https://addons.mozilla.org/en-US/firefo ... e-manager/
Use Tools > SQLite Manager
Top right select places.sqlite and click Go
Under Tables select moz_annos
On the Browse & Search tab:
The anno_attribute_id column holds 16,17,18 for the three status rows with same place_id per download
16 = download file, 17 = file name, 18 = status
On the Export Wizard tab export in various CSV formats

If you want to convert the download endTime to Windows date-time use:
= ( endTime / 86400000 ) + 25569
i.e. ( Time / millisecs per day ) + days from 1/1/1900 to 1/1/1970
86400000 = 24 x 60 x 60 x 1000 and 25569 = ~ 70 x 365.27
where 1/1/1900 is Windows base date, and 1/1/1970 is Unix base date

[jmurphy]

Okay, Mike, I thought I was following you okay, and then I hit the wall.

sqlite-manager has been installed and appears on the Tools menu.
I invoked it and it launched okay.
I selected places.sqlite and clicked Go.
I went to Tables and seleted moz_annos which is the second in the list on the right after moz_anno_attributes.
I am looking on the Browse & Search Tab.

Here's where I get lost. The anno_attribute_id column has a value of 7 in every row.
I also don't see an Export Wizard tab. I have tabs that say 'Structure', 'Browse & Search', 'Execute SQL' and 'DB settings'. Then there are a lot of buttons underneath to search, edit, delete, and so on.

Did I get the wrong add-on? This one is developed by lazierthanthou and it's Version 0.8.1 updated on September 22, 2013.

The FAQ on his website says:
---- quoted ----
Export can be accessed in these ways:

1. from the menu: Table/View -> Export Table/View
2. Structure tab -> Export button
3. right click on a table/view in the tree on the left -> Export Table/View in the context menu
--- end quote ---


But it also says there that he is in the process of converting the FAQ to help and that his site may not be up to date.

I do see an Export button on the Structure tab.

Firefox is 33.0.2 at the moment. I haven't updated to 33.0.3 yet.

Thanks for your help.

[tatewise]

BTW: There is no need to quote the previous posting when replying.

I am also running Firefox 33.0.2 and you have the correct Add-on.

I had not spotted that it isn't until you use Table > Export Table that you get the Export Wizard tab ~ sorry! The Structure tab Export button does the same.

The anno_attribute_id does seem to vary, and on the PC I am using now with Firefox 33.0.2 the download values are 10,11,12 but there are other values for other records. The moz_anno_attributes table explains the values.

At the bottom of the Browse & Search tab is says what records are shown and the total. Use the >> button to show the latest records, which should include your latest downloads. Compare that with the Firefox Show all downloads history library. When you use Export Wizard all the records are exported.

To filter the records use Search button:
In field 3. anno_attribute_id (INTEGER) select custom
Click blue arrow to right and enter:
>= 10 AND anno_attribute_id <= 12 then click OK