Topic: Upload Takes 10 Minutes At Remote Manifest Update Stage

Firstly, thank you Alfred for setting up this super forum.

When I upload a page to the server and the upload has almost completed, the program 'hangs' for around 10 minutes before confirm that the files were successfully uploaded. This never used to happen. Is there anything I can do to speed up this process?

2 (edited by ZOSman 2022-08-29 21:07:52)

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

For most uploads I use FileZilla. Unless an image (picture) is regenerated by WP I can upload any page that I published to disk with FileZilla. It literally uploads in about a second. I have uploaded hundreds of pages at a time in a fraction of the time that WP publish to web takes.

I also have WP generate an 'updated' date at the bottom of a page so I can easily see that the latest page was updated online when I check to make sure the uploaded file still looks right (sometimes the preview looks fine but what I see online doesn't look the same).

I mention images because they are the only situations that I use WP publish to disk (and occasionally for an untended overnight publish all pages (with delete to get rid of any old unused files). I'm not sure what the problem with images is but I had a problem with this in the past.

LenN

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

The upload speed is fine. It is not until it reaches the final stage that the lag occurs. This may well be linked to the size of the site as it used to be fine.

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

I know exactly what you mean. it seems like it takes forever to 'update remote manifest' etc. That's why I only use WP publish to web when I have to. I can upload hundreds of files with FileZilla (or probably any other good FTP client) in the time it takes WP to upload 10.

LenN

5 (edited by pberk 2022-09-02 15:51:55)

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

LenN wrote:

I know exactly what you mean. it seems like it takes forever to 'update remote manifest' etc. That's why I only use WP publish to web when I have to. I can upload hundreds of files with FileZilla (or probably any other good FTP client) in the time it takes WP to upload 10.

My experience is that the upload actually take place nearly as quick as FileZilla but that the "update remote manifest" takes forever. So, one can cancel the upload during "update remote manifest" and all is well. 

The question is how important is "update remote manifest"?  Does it do anything useful?  Can you skip it?

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

pberk wrote:
LenN wrote:

I know exactly what you mean. it seems like it takes forever to 'update remote manifest' etc. That's why I only use WP publish to web when I have to. I can upload hundreds of files with FileZilla (or probably any other good FTP client) in the time it takes WP to upload 10.

My experience is that the upload actually take place nearly as quick as FileZilla but that the "update remote manifest" takes forever. So, one can cancel the upload during "update remote manifest" and all is well. 

The question is how important is "update remote manifest"?  Does it do anything useful?  Can you skip it?

I am reluctant to cancel unless I can be sure that there are no significant consequences. When it says 'Updating remote manifest' it still states that there are 3 seconds remaining. I am not sure what processes are going on during that supposed 3 seconds.

7 (edited by pberk 2022-09-06 23:14:22)

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

Bob Keen wrote:
pberk wrote:
LenN wrote:

I know exactly what you mean. it seems like it takes forever to 'update remote manifest' etc. That's why I only use WP publish to web when I have to. I can upload hundreds of files with FileZilla (or probably any other good FTP client) in the time it takes WP to upload 10.

My experience is that the upload actually take place nearly as quick as FileZilla but that the "update remote manifest" takes forever. So, one can cancel the upload during "update remote manifest" and all is well. 

The question is how important is "update remote manifest"?  Does it do anything useful?  Can you skip it?

I am reluctant to cancel unless I can be sure that there are no significant consequences. When it says 'Updating remote manifest' it still states that there are 3 seconds remaining. I am not sure what processes are going on during that supposed 3 seconds.

It's never done me any harm to cancel.  I usually let it run to completion later on, when it is convenient for me.  I've never found anyone who can tell me why it would be important to do the "updating remote manifest".  It seems clear to me that you don't need to do it every time you update your site.  So if you are worried, do it when have the time.

By the way, I have a very large site .. about 5000 images and hundreds of pages. It takes "updating remote manifest" about an hour to run.

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

Since there is no option but to do it (either immediately or later) I will continue to do as I have, which is to leave it to do it and that way I know it's been done. I have never been one to put off to tomorrow what I can do today as I am likely to forget. I find it reassuring to have got a task out of the way where I don't have to worry about it again.

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

On your host server you will see a file named WebPlusDb.txt. My undertsanding is that WP uses this file to decide which files need uploading when you update (incremental update) or add a new page to your site. Over time this file seems to become somewhat bloated - when I looked at mine a few days ago it was something like 14Mb - and the update becomes ever more time consuming. My workaround is to periodically rename WebPlusDb.txt as WebPlusDb.old. (This is a backup in case anything goes wrong.)

WebPlusDb.txt no longer exists on the server so WP rebuilds it progressively as you make changes or additions to your site. My current manifest file size is just over 800 bytes as I've only made a few changes to the website. The downside is that an incremental update will effectively be a full update first time round but the gain is that the manifest update is almost instantaneous.

Hope this helps.

10 (edited by Bob Keen 2022-11-07 11:04:31)

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

How do I rename this file? Is it simply a case of going into Maintain Site and then changing the file name in the tree that appears?

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

Bob Keen wrote:

How do I rename this file? Is it simply a case of going into Maintain Site and then changing the file name in the tree that appears?

Yes, that should do the trick.

“Within you, there is a stillness and a sanctuary to which you can retreat at any time and be yourself.”
― Hermann Hesse

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

Thank you Alfred.

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

The remote manifest file is shown at the foot of the window in Maintain Website. I have attempted to amend the file extension but it will not do it. Before I do anything that might be reckless (given my lack of knowledge on these things) would it work if I deleted the entire file and then replace it with the 'Old' extension version?

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

The remote manifest file allows the FTP uploader in WebPlus to work out what it needs to do for an incremental update. If you rename or delete the manifest then the uploader won’t have the necessary information for an incremental update, so it will simply do a full upload instead.

“Within you, there is a stillness and a sanctuary to which you can retreat at any time and be yourself.”
― Hermann Hesse

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

In that case Alfred, could I just delete the contents of the manifest folder, do a full upload, and therefore speed up future updating of the manifest?

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

I’m not sure what you mean, Bob. If you choose to do a full upload, there’s no need for the FTP uploader to work out what needs updating.

“Within you, there is a stillness and a sanctuary to which you can retreat at any time and be yourself.”
― Hermann Hesse

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

Alfred wrote:

I’m not sure what you mean, Bob. If you choose to do a full upload, there’s no need for the FTP uploader to work out what needs updating.

What I want to achieve Alfred is to prevent the time it is taking for the remote manifest to update each time I upload something. GWYNDY suggested amending the WebPlusDb.txt file to  WebPlusDb.Old in order that webplus would not have to search through lots of previous files to do the update. The downside he said was that the next upload would effectively be a full upload, but the new WebPlusDb.txt file would be empty and therefore update faster.

The problem that I hit was finding a way to amend the WebPlusDb.txt file. I tried to change it in the Maintain file section but it would not permit me to amend the file extension. I just want to avoid a 10 minute plus wait every time I upload even the smallest file.

I hope that explains it.

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

Bob Keen wrote:

The problem that I hit was finding a way to amend the WebPlusDb.txt file. I tried to change it in the Maintain file section but it would not permit me to amend the file extension. I just want to avoid a 10 minute plus wait every time I upload even the smallest file.

I hope that explains it.

Thanks, Bob. You’ve explained it very clearly!

If ‘Maintain Website’ won’t let you do it (for whatever reason) you’ll have to resort to using the file manager in your hosting control panel, or a third-party utility such as FileZilla.

“Within you, there is a stillness and a sanctuary to which you can retreat at any time and be yourself.”
― Hermann Hesse

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

Hi Alfred

I have managed to change the extension to '.old' in the cpanel as that had an option to rename the file. However, when I enter Maintain website in WebPlus, the file still shows as '.txt'. Does this mean that when I upload, a new .txt file will be created which is empty? I assume it would.

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

Sorry I didn’t get back to you yesterday, Bob. As you surmised, renaming WebPlusDb.txt will force WebPlus to create a new file at the next upload attempt. It doesn’t matter whether the file has been renamed to ‘WebPlus.old’, ‘WebPlus.old.txt’ or ‘WebPlus.txt.old’ as long as you can identify it if necessary.

“Within you, there is a stillness and a sanctuary to which you can retreat at any time and be yourself.”
― Hermann Hesse

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

Thank you Alfred. I would not necessarily have expected you to get back to me yesterday as I am sure you have other things to do besides staring at the forum waiting for people like me to pose questions. I am grateful for any response, however long it takes.

Re: Upload Takes 10 Minutes At Remote Manifest Update Stage

Hi all

Just as a point of interest, I did initially get speedier completion of the uploaded page, but since doing a full upload of the site, it has reverted to taking 10 minutes again.
It would seem that there is still some 'memory' in there of the various previous uploads that originally caused the problem.