Solution Manager

1. Get the Denodo Solution Manager installer, license file and certs files from Box (download folders as zip files). The installer folder has been updated to include the latest update as of 2022-08-15.

2. scp the installer, license file, and certs files to the server where you want to install Solution Manager

3. ssh into the server and do the following (all steps beyond this assume you are still in the root shell):

   sudo su unzip denodo-install-solutionmanager-8.0.zip unip certs.zip cd denodo-install-solutionmanager-8.0 chmod +x installer_cli.sh ./installer_cli.sh install

   1. For install path specify /opt/denodo

   2. Use defaults for all remaining options

   3. The installer can take 5-10 minutes to run, make sure your machine is kept awake to keep your session alive.

4. Once the installer is done, copy the license and cert files into the Denodo installation directory

   cp LICENSE_FILE_PATH /opt/denodo/conf/denodo.lic cp certs/* /opt/denodo/conf

    

5. Run the TLS configurator script

   cd /opt/denodo/bin sudo ./denodo_tls_configurator.sh \ --keystore /opt/denodo/conf/denodo_server_key_store.jks \ --cert-cer-file /opt/denodo/conf/denodo_server_public_key.cer \ --truststore /opt/denodo/jre/lib/security/cacerts \ --credentials-file /opt/denodo/conf/credentials.txt \ --denodo-home /opt/denodo

6. Start the Solution Manager services. For a script, see sm_start.sh

7. Confirm the Solution Manager works via a web browser

Virtual Data Port (VDP)

1. Get the Denodo VDP Manager installer, and vdp-conf.zip files from Box (download folders as zip files). The installer folder has been updated to include the latest update as of 2022-08-15. To update the installer for a new update see https://community.denodo.com/docs/html/browse/latest/en/platform/installation/unattended_installation_of_the_denodo_platform/unattended_installation_of_the_denodo_platform#modifying-the-denodo-platform-installer-to-include-the-latest-update

2. scp the installer and certs file to the server where you want to install Solution Manager

3. ssh into the server and do the following (all steps beyond this assume you are still in the root shell):

   1. Unzip the zipped files

   2. Run the install script

      1. For install path use /opt/denodo

      2. For license manager host use denodo-solution-manager-prod-01.it.tufts.edu

      3. The install can take 5-10 minutes to run

4. Once the installer is done, copy the certs files into the Denodo conf directory

5. Configure TLS

   1. Run the TLS Configurator Script

   2. Edit /opt/denodo/conf/SolutionManager.properties and ensure the following line is present and uncommented

       

6. Edit /opt/denodo/vdp/VDBConfiguration.properties so that com.denodo.vdb.vdbinterface.server.VDBManagerImpl.registryURL is set to the hostname of the server.

7. Start the Virtual Data Port service (with Data Catalog). See start.sh

8. Confirm the VDP server is working using a local client or DBeaver.

   1. Design Tool

   2. Scheduler Web Admin

   3. Data Catalog

Updating the Servers

1. Download the applicable update jar from Box or the Denodo support site.

2. scp the jar into the target server

3. Review guidelines outlined here regarding updates: https://community.denodo.com/docs/html/browse/8.0/en/platform/installation/installing_updates_and_hotfixes/installing_updates_and_hotfixes#installing-an-update-or-hotfix-from-the-command-line

4. Stop server. See Starting and Stopping Servers

5. Run the update as the denod user.

6. Restart the server. See Starting and Stopping Servers

Starting and Stopping Servers

The Denodo servers/processes are now running as services under systemd. See this README for more information.

If a stop or restart is taking too long, you can do the following:

SSL Certificates

Leveraging the work of Ned Harvey and team, we have a CA-signed certificate generated by LetsEncrypt automatically delivered to our Denodo servers every 90 days.

Certbot will start attempting to generate and deploy new certificates approximately 1 month prior to the expiration of existing certificates. Generated and deployment attempts are set to take place on Tuesday, Wednesday and Thursday at 6:40 am. With generation and deployment centrally managed in one workflow, there’s little control over the date and time, but there is an option to have the deployment script executed separately which would allow control over the specific day and time for that part of the process.

Open

Example details for a LetsEncrypt certificate for Denodo

Deployment

The certificate files will be deployed by certbot across all Denodo serversm, storing them at /home/certbot/certdir. A cron job (managed by Ned’s team via puppet) runs every Tuesday and Wednesday and Thursday at 5AM which runs /home/denod/denodo/scripts/certbot_install_cert.sh. Certificate deployment will only occur when the certificates are detected in the certbot directory.

All scripts can be found here.

Zero Downtime in Production (WIP)

Leveraging the load balancer in front of our production cluster (WIP), we can take each server out of rotation, let it drain out, update the certificate, then return it into the rotation to prevent any downtime.

Browsers

While most other apps will see the updated certificates immediately, for browsers (like Chrome) you may need to clear your cache or restart the browser for the new certificate to appear.

References

https://community.denodo.com/docs/html/browse/7.0/platform/installation/postinstallation_tasks/enable_ssl_connections_in_the_denodo_platform_servers/obtaining_and_installing_an_ssl_certificate

https://tuftswork.atlassian.net/wiki/spaces/EnterpriseSystems/pages/89457708

https://community.denodo.com/docs/html/browse/8.0/en/platform/installation/postinstallation_tasks/enable_ssl_connections_in_the_denodo_platform_servers/denodo_tls_configurator_script

Load Balancer

The load balancer can be found here: https://tssdc-lb-float-vip.net.tufts.edu/
You need to create a ticket requesting access if you want read or write access.

This page outlines how to create a health check monitor that can be used to remove inactive nodes from a pool: https://community.denodo.com/kb/en/view/document/Configuring%20a%20Denodo%20Cluster%20with%20BigIP%20F5

iRules

We can use iRules to direct the traffic to specific nodes based on URL.

SSH Via Public Key

In order to automate ssh and scp commands to Denodo servers, you need to setup public key authentication for a user (such as denod) on each server.

To do this, do the following:

1. generate an SSH key (ssh-keygen -t ed25519) on the client machine

2. copy the public key .ssh/id_ed25519.pub into ~/.ssh/authorized_keys on the remote machine for the target user.

3. As root/sudo, update /etc/ssh/sshd_config to include an exception for the target user to prevent Duo authentication

4. Reload the sshd_config sudo systemctl reload sshd

Tailscale

Tailscale is a service that helps with service connectivity and encryption.

In the event that Tailscale is causing connectivity issues do the following, which will stop and disable tail scale.

Once Tailscale is confirmed working, do the following to re-enable it and start it back up

JWK

https://community.denodo.com/docs/html/browse/8.0/en/solution_manager/administration/denodo_security_token/denodo_security_token#load-balancer-considerations

1. Update SSO configuration in Solution Manager Web Tool
   
   

   Open

   

2. Update DENODO_HOME/conf/SSOConfiguration.properties. If Solution Manager needs to be accessed from an LB endpoint, set sso.url to reflect the LB URL, and sso.jwt.jwk.sourceUrl to reflect the internal URL pointing to a specific server. This file needs to be updated on all Denodo servers, and requires a server restart after being updated.

3. If using an LB URL, Update DENODO_HOME/conf/vdp/VDBConfiguration.properties to include the property com.denodo.vdb.security.DndTokenAuthenticator.jwkSourceUrl, this will be the same as the sso.jwt.jwk.sourceUrl property above.

Config file examples can be found here.

Usage Statistics

https://community.denodo.com/docs/html/browse/7.0/vdp/data_catalog/administration/personalization/personalization#usage-statistics