Job Array with queueSize exceeds queueSize

[Joon-Klaps]

When combining both executor.queueSize and process.array , I’m reguraly surpassing the executor.queueSize limit is this because the array jobs are seen as a single job?

For example:

process.array = 30
executor.queueSize = 40

The total amount of concurrent jobs running can exceed 60.

Would it be possible to have more control over this?

[bentsherman]

Good catch. I thought we handled this case when we released it but maybe it got lost in some refactoring.

The task polling monitor needs to consider the job array size when deciding whether it can submit a task:

nextflow/modules/nextflow/src/main/groovy/nextflow/processor/TaskPollingMonitor.groovy

Lines 200 to 202 in 0f58857

	protected boolean canSubmit(TaskHandler handler) {
	(capacity>0 ? runningQueue.size() < capacity : true) && handler.canForkProcess() && handler.isReady()
	}

@jorgee can you write a fix for this when you have time