How to increase limit from 20000 variables in REST API in activiti?

Flowable Engine
#1

When making a REST API call to query historic tasks using the parameter “includeProcessVariables”: “true” , a maximum of 20,000 variables are returned. The results of the REST API when using “includeProcessVariables”: “true” are cut off as soon as the 20,000th variable is in the REST API response.

A customer have around 500 process variable in on process and for that he is only able to get the first 40 process Data with includeProcessVariables=true.

In docs , it is mentioned that the limit of 20000 is for performance reason. Can this limit be changed. I am thinking to increase the limit to 100000 or more.

Same issue here https://github.com/flowable/flowable-engine/issues/1348 , https://issues.alfresco.com/jira/browse/ACTIVITI-1065

#2

I haven’t dug too deeply into this code, but my guess is that something like this would work:

    @Bean
    EngineConfigurationConfigurer<SpringProcessEngineConfiguration> processEngineConfigurationEngineConfigurationConfigurer(){
        return engineConfiguration -> {
            System.out.println("configuring Process Engine");
            engineConfiguration.setHistoricTaskQueryLimit(100000);
            engineConfiguration.setHistoricProcessInstancesQueryLimit(100000);
        };
    }

Edit, I just realized the subject says “in Activiti”, are you using Flowable or Activiti?

#3

My bad, I am using flowable.

#4

@wwitt is there a way I can filter only a few process variables while making this call? I am always interested in a few variables not all.

Regards

#5

You’d need to craft some custom queries using the Java APIs, here are some examples from the documentation:


#6

Hi! Is it possible to set those limits with spring properties (in application.properties or with environment variables)?

#7

No, this property isn’t exposed. You’d need a simple EngineConfigurer bean to set it.

#8

@joram what is the property ?

#9

Hi @David_H

Which Flowable OSS version do you use?

This issue is fixed with v6.7.0 (see also Rework how we do paging and including data from other tables by filiphr · Pull Request #2892 · flowable/flowable-engine · GitHub)

Regards,
Simon

#10

We are using Flowable 6.6.0.17 and we are getting the following exception:

logger_name: o.f.j.s.i.a.DefaultAsyncRunnableExecutionExceptionHandler
message: Job JOB-c5678d8d-5bcb-11ed-988d-0a580a822c9a failed
stack_trace: <#7f89484e> j.l.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.util.ArrayList.grow(Unknown Source)
at java.util.ArrayList.grow(Unknown Source)
at java.util.ArrayList.add(Unknown Source)
at java.util.ArrayList.add(Unknown Source)
at o.f.v.s.i.p.e.HistoricVariableInitializingList.add(HistoricVariableInitializingList.java:42)
at o.f.v.s.i.p.e.HistoricVariableInitializingList.add(HistoricVariableInitializingList.java:29)
at o.f.c.e.i.h.HistoricCaseInstanceQueryImpl.addCachedVariableForQueryById(HistoricCaseInstanceQueryImpl.java:579)
at o.f.c.e.i.h.HistoricCaseInstanceQueryImpl.executeList(HistoricCaseInstanceQueryImpl.java:551)
at o.f.c.e.i.q.AbstractQuery.executeSingleResult(AbstractQuery.java:180)
at o.f.c.e.i.q.AbstractQuery.execute(AbstractQuery.java:164)
at o.f.c.e.i.i.CmmnCommandInvoker.lambda$execute$0(CmmnCommandInvoker.java:57)
at o.f.c.e.i.i.CmmnCommandInvoker$$Lambda$1728/0x000000080136dc40.run(Unknown Source)
at o.f.c.e.i.i.CmmnCommandInvoker.executeOperation(CmmnCommandInvoker.java:100)
at o.f.c.e.i.i.CmmnCommandInvoker.executeOperations(CmmnCommandInvoker.java:73)
at o.f.c.e.i.i.CmmnCommandInvoker.execute(CmmnCommandInvoker.java:59)
at o.f.c.e.i.i.TransactionContextInterceptor.execute(TransactionContextInterceptor.java:53)
at o.f.c.e.i.i.CommandContextInterceptor.execute(CommandContextInterceptor.java:105)
at o.f.c.s.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:52)
at o.f.c.e.i.i.LogInterceptor.execute(LogInterceptor.java:30)
at c.f.c.m.i.i.MetricsCommandInterceptor.execute(MetricsCommandInterceptor.java:78)
at o.f.c.e.i.c.CommandExecutorImpl.execute(CommandExecutorImpl.java:56)
at o.f.c.e.i.c.CommandExecutorImpl.execute(CommandExecutorImpl.java:51)
at o.f.c.e.i.q.AbstractQuery.singleResult(AbstractQuery.java:105)
at c.c.c.c.f.a.h.ProcessUtilsImpl.getHistoricCaseInstance(ProcessUtilsImpl.java:89)
at c.c.c.c.f.a.h.AuditModelBuilder.getPayload(AuditModelBuilder.java:161)

Hence I would like to limit the number of variables that the query returns.
Using a newer version of Flowable we don’t get any stack trace but we do run out of memory - I suspect it’s the same cause.

#11

That shouldn’t happen with newer versions. How many variables are you requesting and what API call are you using ?