Hi, I am running Apigee Private Cloud OPDK version 4.18.01 and while enabling monetization service for an organisation, i encounter the below unhandled exception Seed billing documents generation failed for org : dacmont. { "code" : "mint.unhandledException", "message" : "Unanticipated exception. Please contact your Support administrator with code : c4ea1427-a930-44bc-8287-53397bf128e6", "contexts" : [ ] } Details are as below :- #cat /opt/apigee/configmononboardingfile MSIP="172.31.30.164" APIGEE_PORT_HTTP_MS="8080" # Default is 8080. ADMIN_EMAIL="test@example.com" APIGEE_ADMINPW="test12345678" # If omitted, you are prompted for it. # Must use IP addresses for CASS_HOSTS, not DNS names. CASS_HOSTS="172.31.30.164" # Space-separated list IP/DNS names of all Qpid nodes in the data center being configured. QPID_HOST="172.31.30.164" QPID_PORT="8083" # Default is 8083. REGION=dc-1 ORG_NAME="dacmont" # The Edge org where you want to enable monetization. MX_GROUP="mxgroup001" # Default Monetization group. /opt/apigee/apigee-service/bin/apigee-service apigee-provision enable-monetization -f /opt/apigee/configmononboardingfile Checking for required variables Checking required variable MSIP...OK Checking required variable APIGEE_PORT_HTTP_MS...OK Checking required variable ADMIN_EMAIL...OK Checking required variable ORG_NAME...OK Checking required variable CASS_HOSTS...OK Checking required variable MX_GROUP...OK Checking required variable REGION...OK Checking for optional variables Found optional variable APIGEE_ADMINPW...OK --> curl -X GET http://172.31.30.164:8080/v1/servers/self/up [...] --> curl -X GET http://172.31.30.164:8080/v1/analytics/groups/ax/mxgroup001 [...] --> curl -X POST http://172.31.30.164:8080/v1/analytics/groups/ax/mxgroup001 [...] { "code" : "analytics.dataapi.service.GroupAlreadyExists", "message" : "A group with the name mxgroup001 already exists", "contexts" : [ ] } ==> 409 --> curl -X POST http://172.31.30.164:8080/v1/analytics/groups/ax/mxgroup001/properties?propName=consumer-type&propVa... [...] { "name" : "mxgroup001", "properties" : { "consumer-type" : "mint" }, "scopes" : [ "dac~prod" ], "uuids" : { "qpid-server" : [ "ae563ca0-1787-46a4-9c65-42dd1f699ebe" ], "postgres-server" : [ ], "consumer-server" : [ "a5984927-c1b1-4bf8-84ef-9cc991e32d23" ] }, "consumer-groups" : [ ], "data-processors" : { } } ==> 200 Add Qpid Servers to the mxgroup001 --> curl -X GET http://172.31.30.164:8080/v1/servers?type=qpid-server&pod=central®ion=dc-1 [...] response code 200 adding ae563ca0-1787-46a4-9c65-42dd1f699ebe to mxgroup001 as qpid-server --> curl -X POST http://172.31.30.164:8080/v1/analytics/groups/ax/mxgroup001/servers?uuid=ae563ca0-1787-46a4-9c65-42d... [...] Add consumer-servers to the mxgroup001 --> curl -X GET http://172.31.30.164:8080/v1/servers?type=management-server&pod=central®ion=dc-1 [...] response code 200 adding a5984927-c1b1-4bf8-84ef-9cc991e32d23 to mxgroup001 as consumer-server --> curl -X POST http://172.31.30.164:8080/v1/analytics/groups/ax/mxgroup001/servers?uuid=a5984927-c1b1-4bf8-84ef-9cc... [...] Finished setting up mx group Onboarding organization for monetization... --> curl -X GET http://172.31.30.164:8080/v1/mint/organizations/dacmont/sync-organization?childEntities=true [...] --> curl -X GET http://172.31.30.164:8080/v1/organizations/dacmont [...] --> curl -X PUT http://172.31.30.164:8080/v1/organizations/dacmont [...] % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 10 100 10 0 0 480 0 --:--:-- --:--:-- --:--:-- 500 --> curl -X POST http://172.31.30.164:8080/v1/analytics/groups/ax/mxgroup001/scopes?org=dacmont&env=prod [...] { "name" : "mxgroup001", "properties" : { "consumer-type" : "mint" }, "scopes" : [ "dac~prod", "dacmont~prod" ], "uuids" : { "qpid-server" : [ "ae563ca0-1787-46a4-9c65-42dd1f699ebe" ], "postgres-server" : [ ], "consumer-server" : [ "a5984927-c1b1-4bf8-84ef-9cc991e32d23" ] }, "consumer-groups" : [ ], "data-processors" : { } } ==> 200 --> curl -X POST http://172.31.30.164:8080/v1/mint/organizations/dacmont/seed-billing-documents-numbers [...] Seed billing documents generation failed for org : dacmont. { "code" : "mint.unhandledException", "message" : "Unanticipated exception. Please contact your Support administrator with code : c4ea1427-a930-44bc-8287-53397bf128e6", "contexts" : [ ] }
Can you share the management server log? Search for "c4ea1427-a930-44bc-8287-53397bf128e6"
in the system logs
@njoshi I am attaching the management system.log for your reference. Thanks in Advance
@njoshi Please find below the snippet of management server logs for string c4ea1427-a930-44bc-8287-53397bf128e6
cat system-2018-05-19.0.log | grep -i c4ea1427-a930-44bc-8287-53397bf128e6 -C 3 --color=always 2018-05-19 03:21:10,690 main-EventThread INFO ORGANIZATIONS - OrganizationPropertiesWatcher.reAttach() : reattached watcher for path /organizations/dacmont/properties 2018-05-19 03:21:10,693 org:dacmont qtp2033268925-53 INFO c.a.c.i.s.CpsManagementListener - CpsManagementListener.onUpdate() : Update event received for org: dacmont 2018-05-19 03:21:10,767 main-EventThread INFO c.a.a.m.MessageConsumerBuilder - MessageConsumerBuilder.onScopeAdded() : Added group=mxgroup001 scope=ServerScope{organization='dacmont', environment='prod'} 2018-05-19 03:21:10,790 qtp2033268925-45 ERROR MINT.SERVICE - MintExceptionTranslator.generateAndLogUUID() : Unhandled exception thrown. Exception id - c4ea1427-a930-44bc-8287-53397bf128e6 java.lang.NullPointerException: null at com.apigee.mint.repository.EntityManagerWrapper.createEM(EntityManagerWrapper.java:50) ~[mint-1.170821.01-SNAPSHOT.jar:na] at com.apigee.mint.repository.EntityManagerWrapper.<init>(EntityManagerWrapper.java:34) ~[mint-1.170821.01-SNAPSHOT.jar:na] -- -- at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590) [jetty-util-9.4.0.v20161208.jar:9.4.0.v20161208] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162] 2018-05-19 03:21:10,791 qtp2033268925-45 ERROR REST - CustomJAXRSInvoker.performInvocation() : CustomJAXRSInvoker.performInvocation : Method com.sun.proxy.$Proxy119.seedBillingDocumentNumbersForOrganization threw an exception. com.apigee.kernel.exceptions.spi.UncheckedException: Unanticipated exception. Please contact your Support administrator with code : c4ea1427-a930-44bc-8287-53397bf128e6 at com.apigee.mint.exceptions.MintExceptionTranslator.translate(MintExceptionTranslator.java:101) ~[mint-1.170821.01-SNAPSHOT.jar:na] at com.apigee.mint.exceptions.MintExceptionTranslator.translateToApiPlatform(MintExceptionTranslator.java:41) ~[mint-1.170821.01-SNAPSHOT.jar:na] at com.apigee.mint.commons.AbstractExceptionHandlingCapableService.handleException(AbstractExceptionHandlingCapableService.java:79) ~[mint-configs-1.170821.01-SNAPSHOT.jar:na] -- -- at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672) [jetty-util-9.4.0.v20161208.jar:9.4.0.v20161208] at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590) [jetty-util-9.4.0.v20161208.jar:9.4.0.v20161208] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162] 2018-05-19 03:21:10,791 qtp2033268925-45 ERROR REST - ExceptionMapper.toResponse() : Error occurred : Unanticipated exception. Please contact your Support administrator with code : c4ea1427-a930-44bc-8287-53397bf128e6 com.apigee.mint.exceptions.MintExceptionTranslator.translate(MintExceptionTranslator.java:101) com.apigee.mint.exceptions.MintExceptionTranslator.translateToApiPlatform(MintExceptionTranslator.java:41) -- -- org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173) com.apigee.rest.framework.CustomJAXRSInvoker.performInvocation(CustomJAXRSInvoker.java:126) 2018-05-19 03:21:10,791 qtp2033268925-45 ERROR REST - ExceptionMapper.toResponse() : Returning error response : ErrorResponse{errorCode = mint.unhandledException, errorMessage = Unanticipated exception. Please contact your Support administrator with code : c4ea1427-a930-44bc-8287-53397bf128e6} 2018-05-19 03:24:01,001 Taurus_Worker-5 INFO o.q.p.h.LoggingTriggerHistoryPlugin - LoggingTriggerHistoryPlugin.triggerFired() : Trigger DEFAULT.MINT.EMAIL_NOTIFICATION@@@management-server@@@MINT.EMAIL_NOTIFICATION@@@management-server@@@SYSTEM@@@management-server@@@SYSTEM fired job DEFAULT.MINT.EMAIL_NOTIFICATION@@@management-server@@@MINT.EMAIL_NOTIFICATION@@@management-server@@@SYSTEM@@@management-server@@@SYSTEM at: 03:24:01 05/19/2018 2018-05-19 03:24:01,003 Taurus_Worker-5 INFO TAURUS.SCHEDULER.SERVICE - ZKLockManager.createLock() : Acquired lock /taurus/pods/central/locks/lock-mint.scheduler.notificationmp@@@management @ Sat May 19 03:24:01 UTC 2018 : status : true 2018-05-19 03:24:01,003 Taurus_Worker-5 INFO o.q.p.h.LoggingJobHistoryPlugin - LoggingJobHistoryPlugin.jobToBeExecuted() : Job [DEFAULT.MINT.EMAIL_NOTIFICATION@@@management-server@@@MINT.EMAIL_NOTIFICATION@@@management-server@@@SYSTEM@@@management-server@@@SYSTEM] to be fired by trigger [DEFAULT.MINT.EMAIL_NOTIFICATION@@@management-server@@@MINT.EMAIL_NOTIFICATION@@@management-server@@@SYSTEM@@@management-server@@@SYSTEM], re-fire: 0
I went through system log. I believe it is DB connection issue. Can you restart management server in debug mode and attach entire system log after restart here
I am attaching the debug management server logs for your reference.
Please find below the database details :-
psql -h 127.0.0.1 -d apigee -U postgre rkms | _version | table | postgre rkms | address | table | postgre rkms | application | table | postgre rkms | application_category | table | postgre rkms | application_has_product | table | postgre rkms | bank_detail | table | postgre rkms | billing_adjustment | table | postgre rkms | billing_doc_dev | table | postgre rkms | billing_doc_num_generator | table | postgre rkms | billing_doc_product | table | postgre rkms | billing_document | table | postgre rkms | charge_daily | table | postgre rkms | charge_hourly | table | postgre rkms | company | table | postgre rkms | company_category | table | postgre rkms | currency_mapping | table | postgre rkms | custom_attribute | table | postgre rkms | dev_rate_plan_counter | table | postgre rkms | developer | table | postgre rkms | developer_balance | table | postgre rkms | developer_balance_audit | table | postgre rkms | developer_category | table | postgre rkms | developer_has_rate_plan | table | postgre rkms | developer_has_role | table | postgre rkms | developer_role | table | postgre rkms | developer_terms_and_conditions | table | postgre rkms | history | table | postgre rkms | job | table | postgre rkms | limits | table | postgre rkms | limits_counter | table | postgre rkms | mint_locks | table | postgre rkms | monetization_package | table | postgre rkms | notification_action | table | postgre rkms | notification_condition | table | postgre rkms | organization | table | postgre rkms | package_has_product | table | postgre rkms | prod_dev_cat | table | postgre rkms | product | table | postgre rkms | product_broker | table | postgre rkms | provider | table | postgre rkms | rate_plan | table | postgre rkms | rate_plan_details | table | postgre rkms | rate_plan_rates | table | postgre rkms | report_definition | table | postgre rkms | settings | table | postgre rkms | supported_currency | table | postgre rkms | terms_and_conditions | table | postgre rkms | transaction | table | postgre rkms | tx_brokerage | table | postgre rkms | webhook | table | postgre
I am attaching both the silent response configuration file for your reference to install Monetisation service and Monetization Onboarding. (https://docs.apigee.com/private-cloud/v4.18.01/installing-monetization-services)
In "apix-mint.properties" change the config mint.openjpa.log and restart management server
mint.openjpa.log=File=/opt/apigee/var/log/edge-management-server/logs/openjpa.log, Query=ERROR, DefaultLevel=ERROR, Runtime=ERROR, Tool=ERROR
[root@ip-172-31-30-164 conf]# /opt/apigee/apigee-service/bin/apigee-service edge-management-server restart apigee-service: edge-management-server: pid=2032 apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: OK apigee-service: edge-management-server: Not running (DEAD) apigee-service: edge-management-server: OK apigee-configutil: edge-management-server: # OK The file conf/apix-mint.properties was changed [ChangeDelta, position: 68, lines: [#mint.openjpa.log=File=/tmp/openjpa.log, Query=ERROR, DefaultLevel=ERROR, Runtime=ERROR, Tool=ERROR, mint.openjpa.log=File=/opt/apigee/var/log/edge-management-server/logs/openjpa.log, Query=ERROR, DefaultLevel=ERROR, Runtime=ERROR, Tool=ERROR] to [mint.openjpa.log=File=/tmp/openjpa.log, Query=ERROR, DefaultLevel=ERROR, Runtime=ERROR, Tool=ERROR]] apigee-configutil: edge-mint-management-server: # OK apigee-service: edge-management-server: Not running (NO_LOCKFILE) apigee-service: edge-management-server: status=2, continuing apigee-service: edge-management-server: OK apigee-service: edge-management-server: edge-management-server is running [root@ip-172-31-30-164 conf]#
When i restart it gets reset to /tmp/openjpa.log
Change property "conf_apix-mint_mint.openjpa.log" in /opt/apigee/customer/application/mint-management-server.properties
conf_apix-mint_mint.openjpa.log=File=/opt/apigee/var/log/edge-management-server/logs/openjpa.log, Query=ERROR, DefaultLevel=ERROR, Runtime=ERROR, Tool=ERROR
Is that above issue resolved?
Was the issue resolved? Could you please share how? I have similar issue, don't know where to start.