Error while uploading large resource files using APIs

[sarthak]

I am trying to upload a jar file using the resources API which is over 15 MB In size and it is failing. I am getting the following error :

{

  "message" : "com.apigee.datastore.DataAccessException{ code = datastore.ErrorWhileAccessingDataStore, message = Error while accessing datastore;Please retry later, associated contexts = []}",

  "contexts" : [ ],

  "cause" : {

    "code" : "datastore.ErrorWhileAccessingDataStore",

    "message" : "Error while accessing datastore;Please retry later",

    "contexts" : [ ]

  }

The error disappears if the file size is less than 15 MB. So I am guessing somewhere there is a config which restricts file sizes above 15 MB.

?\

This is OPDK 15.04. Anyone knows what I need to do here ?

Update : I have changed the property cassandra.defaults.thrift.transportSizeInMB=15 in conf/apigee/management-server/repository.properties from the default value of 15 to 150 , and restarted Edge, but it did not make a difference.

Update 2: I updated thrift_framed_transport_size_in_mb in conf/cassandra/cassandra.yaml and restarted and it worked.

[sarthak]

UPDATE: Corresponding to the error which the API returns this error is in var/log/apigee/management-server/logs/system.log

qtp1090350296-43 ERROR DATASTORE.CASSANDRA - AstyanaxCassandraClient.insertColumnValueAsBytes() : Adding entry with rowkey org1 to the columnFamily : organizations in the keyspace apprepo failed with {}

com.netflix.astyanax.connectionpool.exceptions.TransportException: TransportException: [host=None(0.0.0.0):0, latency=267(267), attempts=1]org.apache.thrift.transport.TTransportException: java.net.SocketException: Broken pipe

	at com.netflix.astyanax.thrift.ThriftConverter.ToConnectionPoolException(ThriftConverter.java:197) ~[astyanax-thrift-1.56.43.jar:na]

	at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:65) ~[astyanax-thrift-1.56.43.jar:na]

	at com.netflix.astyanax.thrift.AbstractOperationImpl.execute(AbstractOperationImpl.java:28) ~[astyanax-thrift-1.56.43.jar:na]

	at com.netflix.astyanax.thrift.ThriftSyncConnectionFactoryImpl$ThriftConnection.execute(ThriftSyncConnectionFactoryImpl.java:151) ~[astyanax-thrift-1.56.43.jar:na]

	at com.netflix.astyanax.connectionpool.impl.AbstractExecuteWithFailoverImpl.tryOperation(AbstractExecuteWithFailoverImpl.java:119) ~[astyanax-core-1.56.43.jar:na]

	at com.netflix.astyanax.connectionpool.impl.AbstractHostPartitionConnectionPool.executeWithFailover(AbstractHostPartitionConnectionPool.java:338) ~[astyanax-core-1.56.43.jar:na]

	at com.netflix.astyanax.thrift.ThriftKeyspaceImpl.executeOperation(ThriftKeyspaceImpl.java:493) ~[astyanax-thrift-1.56.43.jar:na]

	at com.netflix.astyanax.thrift.ThriftKeyspaceImpl.access$000(ThriftKeyspaceImpl.java:79) ~[astyanax-thrift-1.56.43.jar:na]

	at com.netflix.astyanax.thrift.ThriftKeyspaceImpl$1.execute(ThriftKeyspaceImpl.java:123) ~[astyanax-thrift-1.56.43.jar:na]

	at com.apigee.datastore.client.astyanax.AstyanaxCassandraClient.insertColumnValueAsBytes(AstyanaxCassandraClient.java:662) ~[datastore-1.0.0.jar:na]

	at com.apigee.services.repository.nosql.CassandraRepositoryDelegate.createOrUpdateColumnFamily(CassandraRepositoryDelegate.java:319) [repository-impl-1.0.0.jar:na]

	at com.apigee.services.repository.nosql.CassandraRepositoryDelegate.createOrUpdateIntoOrganizationsColumnFamily(CassandraRepositoryDelegate.java:276) [repository-impl-1.0.0.jar:na]

	at com.apigee.services.repository.nosql.CassandraRepositoryDelegate.store(CassandraRepositoryDelegate.java:245) [repository-impl-1.0.0.jar:na]

	at com.apigee.services.repository.nosql.CassandraRepository.store(CassandraRepository.java:50) [repository-impl-1.0.0.jar:na]

	at com.apigee.services.repository.RepositoryServiceImpl.store(RepositoryServiceImpl.java:250) [repository-impl-1.0.0.jar:na]

	at com.apigee.messaging.config.beans.ResourceUtil.storeResource(ResourceUtil.java:101) [config-model-1.0.0.jar:na]

	at com.apigee.messaging.config.beans.ResourceFactory.create(ResourceFactory.java:50) [config-model-1.0.0.jar:na]

	at com.apigee.messaging.config.beans.OrganizationImpl.addResource(OrganizationImpl.java:190) [config-model-1.0.0.jar:na]

	at com.apigee.distribution.OrganizationResourceAPIImpl.addResource(OrganizationResourceAPIImpl.java:61) [application-distribution-1.0.0.jar:na]

	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.7.0_79]

	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) ~[na:1.7.0_79]

	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.7.0_79]

	at java.lang.reflect.Method.invoke(Method.java:606) ~[na:1.7.0_79]

	at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173) [cxf-rt-core-2.4.7.jar:2.4.7]

	at com.apigee.rest.framework.CustomJAXRSInvoker.performInvocation(CustomJAXRSInvoker.java:125) [rest-1.0.0.jar:na]

	at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:89) [cxf-rt-core-2.4.7.jar:2.4.7]

	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:165) [cxf-rt-frontend-jaxrs-2.4.7.jar:2.4.7]

	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:241) [cxf-rt-frontend-jaxrs-2.4.7.jar:2.4.7]

	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:92) [cxf-rt-frontend-jaxrs-2.4.7.jar:2.4.7]

	at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58) [cxf-rt-core-2.4.7.jar:2.4.7]

	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) [na:1.7.0_79]

	at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_79]

[#3]

Hi @sarthak , thats the default frame size

Can you change the below

# Frame size for thrift (maximum message length).
thrift_framed_transport_size_in_mb: 15 

in cassandra.yaml and check it again ? I am not sure if this is recommended but I believe it should work once you make the change .

Let us know what you find .

[sarthak]

@Maruti Chand

yes , thats what I kind of figured out... I updated my original question with the solution. I changed at two places in cassandra.yaml and in conf/apigee/management-server/repository.properties, and it worked. Thanks for following up.