Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Report outdated / end-of-life Scan Engine / Environment for version 21.4.3 #69

Closed
yu210148 opened this issue Mar 19, 2022 · 40 comments
Closed

Report outdated / end-of-life Scan Engine / Environment for version 21.4.3 #69

yu210148 opened this issue Mar 19, 2022 · 40 comments
Assignees
@yu210148

Comments

@yu210148
Copy link
Owner

Hi all,

Noticed a message about the EOL of scan engine 21.4.3 in my results this morning. I'll test out a fix for the script to get it to pull the "Latest available openvas-scanner version: 21.4.4" and merge it in once I get a chance.

kev.
@yu210148
Copy link
Owner Author

Quick update: seems this is not as simple as updating the version numbers in the script. When I do the install step for gvm-libs fails with permission denied on /var/run. Seems the flag to install it to the different directory is no longer being respected for some reason that I haven't had a chance to investigate yet. Of course, with gvm-libs failing to install the other builds trust depend on it are failing.

I'll keep this thread updated as I work on this.

@yu210148
Copy link
Owner Author

Switched the builds to do their make install step as root and they seem to compile now in the issue 69 branch but the configure openvas step of the script is now failing. I'll need to do more investigation to find a fix for this.

@yu210148
Copy link
Owner Author

Ugg, okay, so got the above configure working again but it looks like the installation instructions for gsa suggest it's been changed considerably for version 21.4.4 from 21.4.3.

@yu210148
Copy link
Owner Author

How a change like this can be in a point release is beyond me.... 🙂

@yu210148
Copy link
Owner Author

This is going to take me some time to work out. I need to work out the new yarn based install for gsa. Preliminary testing shows it's a yarn command then a yarn build, then copy some files. With family commitments I don't have the time to devote to this that I'd like. If anyone smarter than me can take a look at the issue69 branch and send (a) pull request(s) that would be welcome. Otherwise I'll keep at it and post when I have more to share.

@yu210148
Copy link
Owner Author

yu210148 commented Mar 22, 2022 •
edited
Loading

Okay, looks like I've got yarn building gsa now in the issue69 branch. I get the following warnings (see below) during 'yarn build' but they don't seem to be fatal to the build process. I'm not particularly confident that this is going to work when it's done without more tweaking but it's certainly not going to work if it doesn't build so this is progress.

Next up there seems to be permission trouble running /opt/gvm/bin/gvm-manage-certs -a as the gvm user. Not sure if it would be better to run it as root or change the permissions/ownership of the directories but I wanted to note it here.

Here are the warning messages yarn build is showing me:
warning "@greenbone/ui-components > [email protected]" has unmet peer dependency "[email protected] - 3". warning "@greenbone/ui-components > [email protected]" has unmet peer dependency "popper.js@^1.16.1". warning "@greenbone/ui-components > [email protected]" has unmet peer dependency "react-is@>= 16.8.0". warning " > [email protected]" has unmet peer dependency "webpack@>=2". warning "react-scripts > @typescript-eslint/eslint-plugin > [email protected]" has unmet peer dependency "typescript@>=2.8.0 || >= 3.2.0-dev || >= 3.3.0-dev || >= 3.4.0-dev || >= 3.5.0-dev || >= 3.6.0-dev || >= 3.6.0-beta || >= 3.7.0-dev || >= 3.7.0-beta". warning "@storybook/react > [email protected]" has unmet peer dependency "typescript@>= 3.x". warning "@storybook/react > react-docgen-typescript-plugin > [email protected]" has unmet peer dependency typescript@>= 3.x". warning "@storybook/react > react-docgen-typescript-plugin > [email protected]" has unmet peer dependency "typescript@*". warning " > @testing-library/[email protected]" has unmet peer dependency "@testing-library/dom@>=7.21.4". warning " > [email protected]" has unmet peer dependency "eslint@>=7.0.0".

@yu210148
Copy link
Owner Author

I suspect I'll need to add in the nodejs install as per the README at https://github.com/greenbone/gsa

Just want to note this for when I'm looking at why things aren't working later ;)

@GORGES
Copy link
Contributor

GORGES commented Mar 22, 2022

Kevin, I just want to give you encouragement that your script is GREAT and I look forward to your excellent work in fixing it for 21.4.4. I tried myself to work around the /var/run/ directory problem, but it did not work, so I will again rely on your expertise. I did a blog article based on your script a year ago (https://gorges.us/blog/greenbone-vulnerability-scanner-setup/) referring to your script. Kudos!

@yu210148
Copy link
Owner Author

yu210148 commented Mar 22, 2022 via email

@yu210148 yu210148 self-assigned this Mar 22, 2022
@yu210148
Copy link
Owner Author

The script in the issue69 branch now completes up to the "Start OpenVAS Scanner, GSA and GVM services" bit which I'm having a look at. There's still more to do but at least it's building now.

@yu210148
Copy link
Owner Author

Ya, faliing in the "Start OpenVAS Scanner, GSA and GVM services" section with:

Successfully built ospd-openvas Installing collected packages: redis, paramiko, lxml, pyparsing, packaging, ospd-openvas Attempting uninstall: paramiko Found existing installation: paramiko 2.6.0 Not uninstalling paramiko at /usr/lib/python3/dist-packages, outside environment /usr Can't uninstall 'paramiko'. No files were found to uninstall. Attempting uninstall: lxml Found existing installation: lxml 4.5.0 Not uninstalling lxml at /usr/lib/python3/dist-packages, outside environment /usr Can't uninstall 'lxml'. No files were found to uninstall. Successfully installed lxml-4.8.0 ospd-openvas-21.4.4 packaging-20.9 paramiko-2.10.3 pyparsing-3.0.7 redis-3.5.3 export PYTHONPATH=/opt/gvm/lib/python3.8/site-packages /usr/bin/python3 /opt/gvm/bin/ospd-openvas --pid-file /opt/gvm/var/run/ospd-openvas.pid --log-file /opt/gvm/var/log/gvm/ospd-openvas.log --lock-file-dir /opt/gvm/var/run -u /opt/gvm/var/run/ospd.sock /opt/gvm/sbin/gvmd --osp-vt-update=/opt/gvm/var/run/ospd.sock sudo /opt/gvm/sbin/gsad sleep 10 Traceback (most recent call last): File "/opt/gvm/bin/ospd-openvas", line 8, in <module> sys.exit(main()) File "/opt/gvm/lib/python3.8/site-packages/ospd_openvas/daemon.py", line 1386, in main daemon_main('OSPD - openvas', OSPDopenvas) File "/opt/gvm/lib/python3.8/site-packages/ospd/main.py", line 103, in main init_logging( File "/opt/gvm/lib/python3.8/site-packages/ospd/logger.py", line 98, in init_logging fileConfig(config, disable_existing_loggers=False) File "/usr/lib/python3.8/logging/config.py", line 79, in fileConfig handlers = _install_handlers(cp, formatters) File "/usr/lib/python3.8/logging/config.py", line 145, in _install_handlers h = klass(*args, **kwargs) File "/usr/lib/python3.8/logging/__init__.py", line 1147, in __init__ StreamHandler.__init__(self, self._open()) File "/usr/lib/python3.8/logging/__init__.py", line 1176, in _open return open(self.baseFilename, self.mode, encoding=self.encoding) FileNotFoundError: [Errno 2] No such file or directory: '/opt/gvm/var/log/gvm/ospd-openvas.log' sudo: /opt/gvm/sbin/gsad: command not found

@yu210148
Copy link
Owner Author

Something odd when trying to start GVM in the script the /opt/gvm/sbin path doesn't exist. I'll need to check into this further.

@yu210148
Copy link
Owner Author

Dealt with an issue in my testing environment this morning where /tmp had been removed. Got that sorted.

The latest seems to be that the line that executes:
/usr/bin/python3 /opt/gvm/bin/ospd-openvas --pid-file /opt/gvm/var/run/ospd-openvas.pid --log-file /opt/gvm/var/log/gvm/ospd-openvas.log --lock-file-dir /opt/gvm/var/run -u /opt/gvm/var/run/ospd.sock

is failing with:
ModuleNotFoundError: No module named 'ospd_openvas'

@yu210148
Copy link
Owner Author

Looks like this may have something to do with $PYTHONPATH not being exported. When I did it manually it still fails but with something different:

Traceback (most recent call last):
File "/opt/gvm/bin/ospd-openvas", line 8, in
sys.exit(main())
File "/opt/gvm/lib/python3.8/site-packages/ospd_openvas/daemon.py", line 1386, in main
daemon_main('OSPD - openvas', OSPDopenvas)
File "/opt/gvm/lib/python3.8/site-packages/ospd/main.py", line 103, in main
init_logging(
File "/opt/gvm/lib/python3.8/site-packages/ospd/logger.py", line 98, in init_logging
fileConfig(config, disable_existing_loggers=False)
File "/usr/lib/python3.8/logging/config.py", line 79, in fileConfig
handlers = _install_handlers(cp, formatters)
File "/usr/lib/python3.8/logging/config.py", line 145, in _install_handlers
h = klass(*args, **kwargs)
File "/usr/lib/python3.8/logging/init.py", line 1147, in init
StreamHandler.init(self, self._open())
File "/usr/lib/python3.8/logging/init.py", line 1176, in _open
return open(self.baseFilename, self.mode, encoding=self.encoding)
FileNotFoundError: [Errno 2] No such file or directory: '/opt/gvm/var/log/gvm/ospd-openvas.log'

@yu210148
Copy link
Owner Author

If I manually create that file/path it fails with the following:

gvm@gvm21:~/var/log/gvm$ /usr/bin/python3 /opt/gvm/bin/ospd-openvas --pid-file /opt/gvm/var/run/ospd-openvas.pid --log-file /opt/gvm/var/log/gvm/ospd-openvas.log --lock-file-dir /opt/gvm/var/run -u /opt/gvm/var/run/ospd.sock
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/opt/gvm/lib/python3.8/site-packages/ospd/main.py", line 83, in exit_cleanup
server.close()
File "/opt/gvm/lib/python3.8/site-packages/ospd/server.py", line 231, in close
super().close()
File "/opt/gvm/lib/python3.8/site-packages/ospd/server.py", line 149, in close
self.server.shutdown()
AttributeError: 'NoneType' object has no attribute 'shutdown'

@yu210148
Copy link
Owner Author

I suspect the answer to this may be in https://github.com/greenbone/ospd-openvas. I'll investigate when I can.

@GORGES
Copy link
Contributor

GORGES commented Mar 25, 2022

Kevin, I'm monitoring your progress, and appreciate your work!! Thank you!!

@schneemass
Copy link

schneemass commented Mar 25, 2022 •
edited
Loading

Hi,
maybe two links that will help you:
https://greenbone.github.io/docs/gvm-21.04/index.html#ospd-openvas
-- the complete manual installation works fine :-)

https://community.greenbone.net/t/gvm-release-version-21-4-4/11506
-- Current version numbers of the source code releases and repository links:
gsa v21.4.4
gsad v21.4.4
gvmd v21.4.5
gvm-libs v21.4.4
openvas-scanner v21.4.4
ospd-openvas v21.4.4

Best greetings from Allgäu

@fpfbabes
Copy link

I suspect the answer to this may be in https://github.com/greenbone/ospd-openvas. I'll investigate when I can.

Hi Kevin,
I'm monitoring your progress, and appreciate your work!!
im stuck with this new build as well

Thank you!!

@yu210148
Copy link
Owner Author

I've updated the gvmd version and added in additional package dependencies that were listed in the greenbone repositories. Also fixed a few other bugs that I introduced while doing that; however, the original issue with the python traceback persists. I'll continue reading and troubleshooting as time permits. Just wanted to note that I haven't given up on this yet. Oh, I also added a note in the readme. Hopefully anyone finding this will see it before they end up installing an EOL version :)

@yu210148
Copy link
Owner Author

Just rebuilt the thing manually this morning using the steps at https://greenbone.github.io/docs/gvm-21.04/index.html--Thanks schneemass, and while it doesn't have an encrypted https connection to the web interface it does work. It's not immediately clear to me what is different between what this script is doing and what I did manually however, it does give me something to go on. I'm thinking what I'll do is re-write the thing based on those instructions. It'll break the version 20 install but I doubt anyone would want to use that anyway.

If I can manage to get it working that way I'll then see if I can sort out what's involved to get it to use https. If anyone has any thoughts on that and can post them that would be helpful.

Thanks everyone.

kev.

@GORGES
Copy link
Contributor

GORGES commented Mar 28, 2022

Hi - I listed the steps that I took to add an SSL certificate to the community install in a blog article a year ago:

https://gorges.us/blog/greenbone-vulnerability-scanner-setup/

Thank you for your hard work, Kev! I will probably attempt the manual install within a week or so.

@yu210148
Copy link
Owner Author

yu210148 commented Mar 28, 2022 via email

@fpfbabes
Copy link

fpfbabes commented Mar 29, 2022 via email

@Chris-P-BirdDog
Copy link

Hi Kevin - Thank you for all your work on this - It is most appreciated!
Wondering if this could help for installing Postgres 11 - As opposed to getting 12 to work :
https://www.howtodojo.com/install-postgresql-11-on-ubuntu-20-04/

Cheers.

@yu210148
Copy link
Owner Author

Hi all,

I don't think I ran into the 'libical' issue above and it seemed to work for me manually with posgresql 12 in my testing environment when I did it manually. I've had a bit of time this morning and done a re-write in the branch for this issue. I'm just testing it for the first time in a VM. There are a couple of bugs so far involving the 'su'ing' to different users.

kev.

@fpfbabes
Copy link

fpfbabes commented Mar 29, 2022 via email

@yu210148
Copy link
Owner Author

yu210148 commented Mar 29, 2022 via email

@brettahaines
Copy link

@fpfbabes If you are installing manually, check out the link below since the official directions look to have typos around postgresql 11 to 13 on Debian 11 and also python-impacket to python3-impacket

https://community.greenbone.net/t/discussion-gvm-release-version-21-4-4/11785/21

@yu210148
Copy link
Owner Author

yu210148 commented Mar 29, 2022 via email

@fpfbabes
Copy link

fpfbabes commented Mar 30, 2022 via email

@yu210148
Copy link
Owner Author

yu210148 commented Mar 30, 2022 via email

@Chris-P-BirdDog
Copy link

Hi - Tested the new script and it worked - Was playing Def Leppard at the time - Not sure if that helped the build, but just in case it did :)

Ran a test scan and it completed without the nagging "EoL" vuln.
Do think its strange that from the UI / Help / About, it lists the version as: Version 21.4.3
Also - Within UI / Admin / Feed Status, the GVMD_DATA feed is Too Old, 20220128T1556 (over 60 days old) - The rest of the feeds are current.

Thanks again, Kevin!

@yu210148
Copy link
Owner Author

yu210148 commented Mar 31, 2022 via email

@Chris-P-BirdDog
Copy link

It was mostly the Hysteria / Pyromania time range.

Chris

@yu210148
Copy link
Owner Author

I ran into Issue 26 again with the new setup so I had to create a new scan config but otherwise, it appears to be working. Thanks alexmateescu Going to close this off we can re-open it if needed down the road.

@yu210148 yu210148 mentioned this issue Apr 4, 2022
Closed
@fpfbabes
Copy link

fpfbabes commented Oct 11, 2022 via email

@yu210148
Copy link
Owner Author

yu210148 commented Oct 11, 2022 via email

@yu210148
Copy link
Owner Author

yu210148 commented Oct 11, 2022 via email

@fpfbabes
Copy link

fpfbabes commented Oct 11, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yu210148 yu210148

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@yu210148 @GORGES @brettahaines @Chris-P-BirdDog @fpfbabes @schneemass