Big board Find Orphaned Attachments support

This core tool is broken for big boards. MyConversations fixes that.

First up: broken or not, what does this tool do?

For various reasons, attachments to posts (and conversations) can become "orphaned". This tool facilitates their removal, since they no longer serve a purpose, taking up needless space in your filesystem and/or database. The four types of orphaned attachments occur when the attachment:

1. Exists in the filesystem but not in the database.
2. Exists in the database but not in the filesystem (the converse of the above).
3. Is attached to a post or conversation message that no longer exists.
4. Was uploaded more than a day ago by a member in the course of composing a post or conversation message which the member did not end up submitting.

OK, so, how is this tool broken in core for big boards?

In trying to find all orphans at once, using two steps in sequence without stopping (the first handling type one above, and the second handling the remaining three), resource limits are exceeded on big boards which contain many attachments (the big board on which this feature was tested contained over 400,000 attachments). On such boards, the tool often never completes its two-step detection phases, and, even when it does, the tool's deletion phase for a very large number of orphans also typically exceeds resource limits, such that the detected orphaned attachments can't be deleted anyway.

Via which means, then, does MyConversations fix this tool for big boards?

1. Pagination. Rather than trying to detect/delete all orphaned attachments in one go, a configurable per-page orphan limit is supplied from the start. The (very strongly) suggested maximum for this limit is based upon the web server's php.ini max_input_vars setting, because, when each page of orphaned attachments is presented as a list of checkable items to be deleted, each checkbox represents an input variable, and, should all checkboxes remain checked, their count plus the count of supporting hidden variables needs to be less than this limiting php.ini setting.
2. Resumption. When the "Delete Checked Orphans Then Show Next Page" button is clicked, MyConversations - to the extent possible - resumes orphan detection where the previous page left off. The caveat re possible extent is that it is not strictly possible to resume detection of the first type of orphan (see above) at the point at which the last page left off: instead, the list of attachment files in the filesystem is rapidly traversed until the point at which detection for the last page left off is reached, and then detection resumes.
3. Indexation. MyConversations offers the admin the chance to create a database index which significantly speeds up the deletion step, and may even have sped up step two. On the test board, this reduced deletion time from over four minutes to under five seconds.
4. Improved filesystem scanning. MyConversations vastly simplifies the code which scans for attachments in the filesystem, and this simplified code seems to simultaneously speed up that process a great deal.

OK, but what if I don't want to switch from private messages to MyConversations just for this fix?

In that case, you are advised to follow this process:

1. In the ACP set Configuration » Settings » Board Online / Offline » Board Closed to "Yes". This will avoid the situation in which, having installed and activated MyConversations, your members are forced to temporarily use it rather than the core private messaging system. This results in temporarily preventing them from using your board altogether.
2. Install and activate MyConversations.
3. Run the Find Orphaned Attachments tool (in the ACP under Forums & Posts).
4. Deactivate and optionally uninstall MyConversations.
5. Reverse the above step one by setting Board Closed to "No".

Though that might work, can you please separate out this functionality into a discrete plugin with no MyConversations support and without a need to close the board?

Yes, but only if you are willing to sponsor it financially. Please feel free to contact me if you are.

Inspiration

The need for big board support for the Find Orphaned Attachments tool was recognised in the Clean up >400K attachments thread. Opportunity, skill, and collaboration go hand in hand. Thank you, HLFadmin.