FUPS: Forum user-post scraper
Enter settings for your XenForo forum
To retrieve your posts: fill in the settings below, optionally after reading the questions and answers below the settings form, then click "Retrieve posts!". A status page will appear, updating progress automatically in a status box. When scraping is complete, the results file(s) will be linked to.
Unconditionally required fields are single-asterisked. Double-asterisked fields indicate that at least one of these fields is required.
Answers to possible questions
How can I know if a forum is a XenForo forum?
Typically, XenForo forums can be identified by the presence of the text "Forum software by XenForo" in the footer of their forum pages. It is possible, however, that these footer texts have been removed by the administrator of the forum. In this case, the only way to know for sure is to contact your forum administrator.
Does the script work with forums using a language other than English?
Yes, or at least, it's intended to: if you experience problems, please contact me.
Which skins are supported?
Whichever skin(s) is/are default for the CivilWarTalk, ECIGS SA and Skeptiko forums. FUPS' XenForo scraping functionality was originally developed as a paid job to extract posts from the CivilWarTalk forum; since then it has been tested on the other two forums and seems to function fine. If you need support for another XenForo skin, feel free to contact me.
How long will the process take?
It depends on how many posts are to be retrieved, and how many pages they are spread across. You can expect to wait roughly one hour to extract and output 1,000 posts.
Are images supported?
Yes when scraping based on "Extract User ID"; no when scraping based on "Forum IDs". In the case of the former: if you check "Scrape images" (checked by default), then images are downloaded along with the posts. If not, then all relative image URLs are converted to absolute URLs, so images will display in the HTML output files so long as you are online at the time of viewing those files.
Is the downloading of attachments supported?
In general, yes, but not yet for XenForo forums.
Why is this script so slow?
So as to avoid hammering other people's web servers, the script pauses for five seconds between each page retrieval.
Are there any resource issues of which I should be aware?
Yes - because this site is hosted on a shared server, I am limited to a fixed and fairly small number of processes, and each run of FUPS requires two processes, one for the background process doing the scraping, and another for the status web page. For most users, too, the number of posts is significant and the process will run for some time. Please, then, limit yourself to one run of the script at a time, and if you change your mind about wanting to run the script after having clicked "Retrieve posts!", then please click the cancellation link.