Update pipelines to use topic channels
Adapting pipelines to use topic channels for version outputs is a straightforward process explained in the steps below:
-
Update the template of the pipeline to the latest version that includes support for topic channels (nf-core tools >=3.5.0).
The main change in the template are the following lines:
$PIPELINE_NAME.nf def topic_versions = channel.topic("versions") .distinct() .branch { entry -> versions_file: entry instanceof Path versions_tuple: true } def topic_versions_string = topic_versions.versions_tuple .map { process, tool, version -> [ process[process.lastIndexOf(':')+1..-1], " ${tool}: ${version}" ] } .groupTuple(by:0) .map { process, tool_versions -> tool_versions.unique().sort() "${process}:\n${tool_versions.join('\n')}" } ch_collated_versions = softwareVersionsToYAML(ch_versions.mix(topic_versions.versions_file)) .mix(topic_versions_string) ch_collated_versions = softwareVersionsToYAML(ch_versions) -
Pull the latest changes made to modules using
nf-core modules update. -
Look for issues in the pipeline code where
versionsis an invalid output of a process. Remove these outputs since these are now handled by the topic channels. The errors will most likely look like this:```console title="nextflow.log" ERROR ~ No such variable: Exception evaluating property 'versions' for nextflow.script.ChannelOut, Reason: groovy.lang.MissingPropertyException: No such property: versions for class: groovyx.gpars.dataflow.DataflowBroadcast ``` Example: We just updated the `samtools/sort` module to use topic channels. The pipeline will most likely have something like this in one of the (sub)workflows: ```groovy title="main.nf" SAMTOOLS_SORT(input) ch_versions = ch_versions.mix(SAMTOOLS_SORT.out.versions) ``` The above error can be fixed by removing the line that references `SAMTOOLS_SORT.out.versions`, as shown below: ```groovy title="main.nf" SAMTOOLS_SORT(input) ch_versions = ch_versions.mix(SAMTOOLS_SORT.out.versions) // [!code --] ```