Running apt-offline but no updates happening #223

mcarvalho-astranis · 2023-09-29T22:44:20Z

mcarvalho-astranis
Sep 29, 2023

Hi Ricky

Thanks for this great package.

Have following instruction on offline upgrade from:

      https://www.debugpoint.com/how-to-update-or-upgrade-ubuntu-offline-without-internet/

Checked apt list --upgradable (on remote host) and shows 26 packages to upgrade.

Have the apt-offline.sig file on the remote host created.

Minor change was done on "set" command: (added --update --upgrade to the line)

     # apt-offline set ./offline-data.sig --update --upgrade

Move to connected computer run "get" command

apt-offline get /tmp/offline-data.sig --bug-reports --threads 5 -d /tmp/Offline-Get/ > /tmp/offline.out 2> /tmp/offline.errors

Issue:

On running get command: (at the connected computer)
I get a lot (235) of correct hits "Downloading http:...."I get too many (105) "ERROR: Giving up on URL...."
I think 105 errors among 340 tries is too many.

Question(1): Can please anyone chip in and let me know if this is expected, or if you see what I possibly doing incorrectly?

Brought Offline-Get/ to remote computer.

Ran apt-offline install

    # apt-offline install Offline-GET/

We see "Proceeding with installation....What would you like to do next: (y, N, ?) Y.....

Exit status is "zero," but we only see three packages returning "synced" and the proces stops

issue:

No update has happened on any of the 26 packages listed as upgradable, although they are seen on the "Downloading" output of the "get" command.

Question(2): What am I doing incorrect?

Please advise.

Many thanks

Marcelo

rickysarraf · 2023-10-02T15:35:47Z

rickysarraf
Oct 2, 2023
Maintainer

Could you please share the full CLI output, along with all the logs ? That'll help ascertain what the problem is.

1 reply

rickysarraf Oct 5, 2023
Maintainer

On running get command: (at the connected computer). I get a lot (235) of correct hits "Downloading http:...."I get too many (105) "ERROR: Giving up on URL...."
I think 105 errors among 340 tries is too many.

Those errors should have been fine. apt-offline cycles through supported archive formats on the remote server, before ultimately giving up. Full console log would have confirmed the same.

See below code:

apt-offline/apt_offline_core/AptOfflineCoreLib.py

Lines 1396 to 1417 in a6cc640

    
           # We could fail with the Packages format of what apt gave us. We can try the rest of the formats that apt or the archive could support 
        
           reallyFailed = True 
        
           for Format in SupportedFormats: 
        
                   NewPackageFile = pkgFileWithType.rstrip(pkgFileWithType.split(".")[-1]).rstrip(".") + "." + Format 
        
                   NewUrl = url.replace(PackageFormat, Format) 
        
                   log.verbose("Retry download %s %s\n" % (NewUrl, LINE_OVERWRITE_FULL) ) 
        
                   #INFO: Why are we doing this? 
        
                   # Because ProgressBar's total_item is fixed 
        
                   # And download_from_web's addItem() increases the active item upon every 
        
                   # cycle through apt's backend archive formats 
        
                   # This ends up resulting in active items being more than total_items 
        
                   # By increasing the counter, the active/total item list is reflected correctly 
        
                   FetcherInstance.items += 1 
        
                   if DownloadPackages(NewUrl, NewPackageFile) is True: 
        
                           reallyFailed = False 
        
                           break 
        
                   else: 
        
                           log.verbose("Failed with URL %s %s\n" % (NewUrl, LINE_OVERWRITE_FULL) ) 
        
                           FetcherInstance.completed() 
        
           if reallyFailed is True: 
        
               log.verbose("Giving up on URL %s %s\n" % (NewUrl, LINE_OVERWRITE_FULL))

I think your problem is the fact that you are running the set command with --update and --upgrade together. While it works, and was the default previously, using those options together is highly discouraged.

Discouraged so much, that I changed those default. See this: 2cab46a

To sum up, I'd encourage you to check out the wiki documentation that has been prepared.

https://github.com/rickysarraf/apt-offline/wiki

mcarvalho-astranis · 2023-10-03T16:38:36Z

mcarvalho-astranis
Oct 3, 2023
Author

Hi Ricky

Many thanks for opening this discussion.

My apologies for not return yesterday. I was scripting my own solution. It worked but I prefer to run with a "institutionalized" package then my own scripts for obvious reason.

I will proceed with "share the full CLI output, along with all the logs." Please give e a couple of hours.

Are there specific logs location I need to be aware off?

Please advise.

Acton Item: From me: "share the full CLI output, along with all the logs."

0 replies

mcarvalho-astranis · 2023-10-04T18:56:01Z

mcarvalho-astranis
Oct 4, 2023
Author

Hi Ricky

Here we go.

Have following instruction on offline upgrade from:

  https://www.debugpoint.com/how-to-update-or-upgrade-ubuntu-offline-without-internet/

I have tried twice. First on a Intel NUC that was online but I faked it offline. Secondly on an offline DEV host that mirrors my production one. By chance I lost the Pen Drive, it died, and I have no logs from the DEV host just the CLI commands and answers. Please bear with me.

       # Creating Signature file - at the offline host

apt-offline set ./sf-moc-changeme_offline-data.sig --update --upgrade

Gathering details needed for 'update' operation
Gathering details needed for 'upgrade' operation

echo $?

0

        # Moving Sig file to Connected host.  I kept the sig file on the Pen Drive and created a directory named the offline host

mkdir sf-moc-changeme

mv sf-moc-changeme_offline-data.sig sf-moc-changeme/

cd sf-moc-changeme/

ls

Sf-moc-changeme_offline-data.sig

           # Creating “Get” directory

mkdir Offline-GET_sf-moc-changeme

         # Getting required packages

apt-offline get ./sf-moc-changeme_offline-data.sig --bug-reports --threads 5 -d ./Offline-GET_sf-moc-changeme/ > offline.out 2> offline.errors

ls

offline.errors  Offline-GET_sf-moc-changeme  offline.out  sf-moc-changeme_offline-data.sig

Attention: here is were I had my offline.out and offline.errors files lost when the Pen Drive died. No harm to the actions that follows

        # Moving “Get” directory to offline mirror


        # Install the downloaded packages to the offline system.
               Pre-install

mkdir Offline-GET

ls /media/xxxxxxxxxxx/Samsung\ USB/MOC/sf-moc-changeme/Offline-GET_sf-moc-changeme/ | wc -l

138                  # This is data from the Pen Drive

ls Offline-GET/ | wc -l

0                      # This is local to the offline host

cp /media/xxxxxxxxxxx/Samsung\ USB/MOC/sf-moc-changeme/Offline-GET_sf-moc-changeme/* Offline-GET/

ls Offline-GET/ | wc -l

138

        # Install the downloaded packages to the offline system
                     Sanity check

apt update

      -----
26 packages can be upgraded. Run 'apt list --upgradable' to see them.

                Go for offline update installs

apt-offline install Offline-GET/

Proceeding with installation
       --------
Following are the list of bugs present.
1051028 google-chrome-stable : google-chrome-stable: Google Chrome does not respect prefers-color-scheme for webpages on KDE Plasma
(Y) Yes. Proceed with installation
(N) No, Abort.
(R) Redisplay the list of bugs.
(Bug Number) Display the bug report from the Offline Bug Reports.
Display this help message.
What would you like to do next: (y, N, ?)Y
libsnmp35_5.8+dfsg-2ubuntu2.6_amd64.deb file synced.
google-chrome-stable_109.0.5414.74-1_amd64.deb file synced.
libsnmp-base_5.8+dfsg-2ubuntu2.6_all.deb file synced.

echo $?

0

     # Validating

apt update

    -----
Reading package lists... Done
Building dependency tree       
Reading state information... Done
26 packages can be upgraded. Run 'apt list --upgradable' to see them.

           Did not work, 26 packages yet to upgrade.

Note: I am attaching .out and .errors files here that ARE NOT from this test as we lost it on the Pen Drive. They are from the first test I did on the NUC and also did not work.

Please advise if you see I am missing something.

One last thing. Is there any logs you can point our to me to take a look?

Many thanks

Marcelo

offline.errors.TXT
offline.out.TXT

1 reply

rickysarraf Oct 5, 2023
Maintainer

�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/main/i18n/Translation-en_US.lzma                                                             
�[1;0m�[1;32m�[1;0m�[1;32m�[1;0m�[1;32m�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/main/dep11/Components-all.yml.lzma                                                             
�[1;0m�[1;32m�[1;0m�[1;32m�[1;0m�[1;32m�[1;0m�[1;32m�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/main/cnf/Commands-all.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/restricted/binary-all/Packages.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/restricted/i18n/Translation-en_US.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/restricted/dep11/Components-amd64.yml.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/restricted/dep11/Components-all.yml.lzma                                                             
�[1;0m�[1;32m�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/restricted/dep11/icons-48x48.tar.lzma                                                             
�[1;0m�[1;32m�[1;0m�[1;32m�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/restricted/dep11/[email protected]                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/restricted/dep11/icons-64x64.tar.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/restricted/cnf/Commands-all.lzma                                                             
�[1;0m�[1;32m�[1;0m�[1;32m�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/universe/binary-all/Packages.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/universe/i18n/Translation-en_US.lzma                                                             
�[1;0m�[1;32m�[1;0m�[1;32m�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/universe/dep11/Components-all.yml.lzma                                                             
�[1;0m�[1;32m�[1;0m�[1;32m�[1;0m�[1;32m�[1;0m�[1;32m�[1;0m�[1;32m�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/universe/cnf/Commands-all.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/multiverse/binary-all/Packages.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/multiverse/i18n/Translation-en_US.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/multiverse/dep11/Components-amd64.yml.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/multiverse/dep11/Components-all.yml.lzma                                                             
�[1;0m�[1;32m�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/multiverse/dep11/icons-48x48.tar.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/multiverse/dep11/icons-64x64.tar.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/multiverse/cnf/Commands-all.lzma                                                             
�[1;0m�[1;31mERROR: Giving up on URL http://us.archive.ubuntu.com/ubuntu/dists/focal-backports/multiverse/dep11/[email protected]                                                             
�[1;0m�[1;32m�[1;0mCalling get_bugs with positional arguments is deprecated, please use keyword arguments instead.
�[1;32m�[1;0m�[1;31mERROR: Some items failed to download. Downloaded data may be incomplete
�[1;0m�[1;31mERROR: Please run in verbose mode to see details about failed items

These errors are fine. They are just complained about the missing metadata payload, which isn't critical to functionality

mcarvalho-astranis · 2023-10-04T19:00:22Z

mcarvalho-astranis
Oct 4, 2023
Author

Ricky

I am sorry it got large fonts. It was not me. It was the "#" prompt I had in front of command lines that provoked that.

Marcelo

0 replies

mcarvalho-astranis · 2023-10-06T14:47:59Z

mcarvalho-astranis
Oct 6, 2023
Author

Hi Ricky Thank you so much. I will go over it the next couple of days and get back here with updates. Many thanks.

…

--

________ This email and any attachments may contain Astranis confidential and/or proprietary information governed by a non-disclosure agreement, and are intended solely for the individual or entity specified by the message.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Running apt-offline but no updates happening #223

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 5 comments 2 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Running apt-offline but no updates happening #223

mcarvalho-astranis Sep 29, 2023

Replies: 5 comments · 2 replies

rickysarraf Oct 2, 2023 Maintainer

rickysarraf Oct 5, 2023 Maintainer

mcarvalho-astranis Oct 3, 2023 Author

mcarvalho-astranis Oct 4, 2023 Author

apt-offline set ./sf-moc-changeme_offline-data.sig --update --upgrade

echo $?

mkdir sf-moc-changeme

mv sf-moc-changeme_offline-data.sig sf-moc-changeme/

cd sf-moc-changeme/

ls

mkdir Offline-GET_sf-moc-changeme

apt-offline get ./sf-moc-changeme_offline-data.sig --bug-reports --threads 5 -d ./Offline-GET_sf-moc-changeme/ > offline.out 2> offline.errors

ls

mkdir Offline-GET

ls /media/xxxxxxxxxxx/Samsung\ USB/MOC/sf-moc-changeme/Offline-GET_sf-moc-changeme/ | wc -l

ls Offline-GET/ | wc -l

cp /media/xxxxxxxxxxx/Samsung\ USB/MOC/sf-moc-changeme/Offline-GET_sf-moc-changeme/* Offline-GET/

ls Offline-GET/ | wc -l

apt update

apt-offline install Offline-GET/

echo $?

apt update

rickysarraf Oct 5, 2023 Maintainer

mcarvalho-astranis Oct 4, 2023 Author

mcarvalho-astranis Oct 6, 2023 Author

mcarvalho-astranis
Sep 29, 2023

Replies: 5 comments 2 replies

rickysarraf
Oct 2, 2023
Maintainer

rickysarraf Oct 5, 2023
Maintainer

mcarvalho-astranis
Oct 3, 2023
Author

mcarvalho-astranis
Oct 4, 2023
Author

rickysarraf Oct 5, 2023
Maintainer

mcarvalho-astranis
Oct 4, 2023
Author

mcarvalho-astranis
Oct 6, 2023
Author