Job Download Web Content Action

Overview

The Job Download Web Content Action node is designed to download assets from the web. To use this node, an additional container is necessary, and it is not included in the default stack. For more details, please refer to the Web-Content-Downloader Stack section.

Configuration

Configuring the node is straightforward. First, you need the IP address of the server host running the Web-Content-Downloader Stack, along with the exposed port (default port: 8010). Next, specify the URL from which the asset will be downloaded, such as "Bundestag - 148. Sitzung." Lastly, you can set a path for saving asset metadata in JSON format. The asset metadata path is optional; if not needed, leave it empty. Metadata will only be downloaded if the source provides sufficient information.

Adjust the job messages and timeout to meet your specific requirements.

Job Download Web Content Action

The node is designed to be utilized with the Cosmo Web_Upload trigger since it provides a suitable input field for web links.

Cosmo Web Upload input field for links

If the link field is not present, verify in the access presets for users and groups whether the Web_Upload_Link has been enabled and/or assigned accordingly.

The node will sequentially download files, one after the other.

Please note that the download speed is dependent on the service host provider and can range from real-time to significantly faster.

Web-Content-Downloader Stack

The setup instructions for the stack required for the Job Download Web Content Action are identical to those for configuring a Linux Client. Please consult the installation instructions provided in the Helmut4 Client article for guidance.

The web-content-downloader can be configured either within the existing Helmut4 stack or on a separate, dedicated host.

Please be aware that the container must have access to specific URLs, which might necessitate adjustments to your local firewall rules.

version: "3.7"

services:    
  web-content-downloader:
    image: repo.moovit24.de:443/web-content-downloader:1.0.1
    restart: always
    
    environment:  
      MAX_CONCURRENT_DOWNLOADS: 25
      MAX_TITLE_LENGTH: 50      

    expose:
      - 8010
    ports:
      - 8010:8010
    volumes:
      - /mnt:/Volumes

Within the configuration, it is possible to set two environment parameters:

1. MAX_CONCURRENT_DOWNLOADS: This integer at the end can be used to define the maximum number of download jobs the container can handle concurrently. If this variable is not set, the default value of 100 will be used.

2. MAX_TITLE_LENGTH: This integer defines the maximum length for the file name generated from the video title. If this variable is not set, the maximum length will default to 80.