mint.unhandledException while enabling monetizatio...

kaushal · 05-18-2018 08:40 PM

Hi,

I am running Apigee Private Cloud OPDK version 4.18.01 and while enabling monetization service for an organisation, i encounter the below unhandled exception

Seed billing documents generation failed for org : dacmont. {
  "code" : "mint.unhandledException",
  "message" : "Unanticipated exception. Please contact your Support administrator with code : c4ea1427-a930-44bc-8287-53397bf128e6",
  "contexts" : [ ]
}


Details are as below :-

#cat /opt/apigee/configmononboardingfile
MSIP="172.31.30.164"
APIGEE_PORT_HTTP_MS="8080"    # Default is 8080.
ADMIN_EMAIL="test@example.com"
APIGEE_ADMINPW="test12345678"    # If omitted, you are prompted for it.
# Must use IP addresses for CASS_HOSTS, not DNS names.
CASS_HOSTS="172.31.30.164"
# Space-separated list IP/DNS names of all Qpid nodes in the data center being configured.
QPID_HOST="172.31.30.164"
QPID_PORT="8083"     # Default is 8083.
REGION=dc-1
ORG_NAME="dacmont"    # The Edge org where you want to enable monetization.
MX_GROUP="mxgroup001"    # Default Monetization group.

/opt/apigee/apigee-service/bin/apigee-service apigee-provision enable-monetization -f /opt/apigee/configmononboardingfile
Checking for required variables
Checking required variable MSIP...OK
Checking required variable APIGEE_PORT_HTTP_MS...OK
Checking required variable ADMIN_EMAIL...OK
Checking required variable ORG_NAME...OK
Checking required variable CASS_HOSTS...OK
Checking required variable MX_GROUP...OK
Checking required variable REGION...OK


Checking for optional variables
Found optional variable APIGEE_ADMINPW...OK
--> curl -X GET http://172.31.30.164:8080/v1/servers/self/up [...]
--> curl -X GET http://172.31.30.164:8080/v1/analytics/groups/ax/mxgroup001 [...]
--> curl -X POST http://172.31.30.164:8080/v1/analytics/groups/ax/mxgroup001 [...]
{
  "code" : "analytics.dataapi.service.GroupAlreadyExists",
  "message" : "A group with the name mxgroup001 already exists",
  "contexts" : [ ]
}
==> 409
--> curl -X POST http://172.31.30.164:8080/v1/analytics/groups/ax/mxgroup001/properties?propName=consumer-type&propVa... [...]
{
  "name" : "mxgroup001",
  "properties" : {
    "consumer-type" : "mint"
  },
  "scopes" : [ "dac~prod" ],
  "uuids" : {
    "qpid-server" : [ "ae563ca0-1787-46a4-9c65-42dd1f699ebe" ],
    "postgres-server" : [ ],
    "consumer-server" : [ "a5984927-c1b1-4bf8-84ef-9cc991e32d23" ]
  },
  "consumer-groups" : [ ],
  "data-processors" : {
  }
}
==> 200
Add Qpid Servers to the mxgroup001
--> curl -X GET http://172.31.30.164:8080/v1/servers?type=qpid-server&pod=central&region=dc-1 [...]
response code 200
adding ae563ca0-1787-46a4-9c65-42dd1f699ebe to mxgroup001 as qpid-server
--> curl -X POST http://172.31.30.164:8080/v1/analytics/groups/ax/mxgroup001/servers?uuid=ae563ca0-1787-46a4-9c65-42d... [...]
Add consumer-servers to the mxgroup001
--> curl -X GET http://172.31.30.164:8080/v1/servers?type=management-server&pod=central&region=dc-1 [...]
response code 200
adding a5984927-c1b1-4bf8-84ef-9cc991e32d23 to mxgroup001 as consumer-server
--> curl -X POST http://172.31.30.164:8080/v1/analytics/groups/ax/mxgroup001/servers?uuid=a5984927-c1b1-4bf8-84ef-9cc... [...]
Finished setting up mx group
Onboarding organization for monetization...
--> curl -X GET http://172.31.30.164:8080/v1/mint/organizations/dacmont/sync-organization?childEntities=true [...]
--> curl -X GET http://172.31.30.164:8080/v1/organizations/dacmont [...]
--> curl -X PUT http://172.31.30.164:8080/v1/organizations/dacmont [...]
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100    10  100    10    0     0    480      0 --:--:-- --:--:-- --:--:--   500
--> curl -X POST http://172.31.30.164:8080/v1/analytics/groups/ax/mxgroup001/scopes?org=dacmont&env=prod [...]
{
  "name" : "mxgroup001",
  "properties" : {
    "consumer-type" : "mint"
  },
  "scopes" : [ "dac~prod", "dacmont~prod" ],
  "uuids" : {
    "qpid-server" : [ "ae563ca0-1787-46a4-9c65-42dd1f699ebe" ],
    "postgres-server" : [ ],
    "consumer-server" : [ "a5984927-c1b1-4bf8-84ef-9cc991e32d23" ]
  },
  "consumer-groups" : [ ],
  "data-processors" : {
  }
}
==> 200
--> curl -X POST http://172.31.30.164:8080/v1/mint/organizations/dacmont/seed-billing-documents-numbers [...]
Seed billing documents generation failed for org : dacmont. {
  "code" : "mint.unhandledException",
  "message" : "Unanticipated exception. Please contact your Support administrator with code : c4ea1427-a930-44bc-8287-53397bf128e6",
  "contexts" : [ ]
}

Report Inappropriate Content

Can you share the management server log? Search for "c4ea1427-a930-44bc-8287-53397bf128e6"

in the system logs

kaushal

@njoshi I am attaching the management system.log for your reference. Thanks in Advance

kaushal

@njoshi Please find below the snippet of management server logs for string c4ea1427-a930-44bc-8287-53397bf128e6

cat system-2018-05-19.0.log | grep -i c4ea1427-a930-44bc-8287-53397bf128e6 -C 3 --color=always

2018-05-19 03:21:10,690  main-EventThread INFO  ORGANIZATIONS - OrganizationPropertiesWatcher.reAttach() : reattached watcher for path /organizations/dacmont/properties
2018-05-19 03:21:10,693 org:dacmont  qtp2033268925-53 INFO  c.a.c.i.s.CpsManagementListener - CpsManagementListener.onUpdate() : Update event received for org: dacmont
2018-05-19 03:21:10,767  main-EventThread INFO  c.a.a.m.MessageConsumerBuilder - MessageConsumerBuilder.onScopeAdded() : Added group=mxgroup001 scope=ServerScope{organization='dacmont', environment='prod'}
2018-05-19 03:21:10,790  qtp2033268925-45 ERROR MINT.SERVICE - MintExceptionTranslator.generateAndLogUUID() : Unhandled exception thrown. Exception id - c4ea1427-a930-44bc-8287-53397bf128e6
java.lang.NullPointerException: null
at com.apigee.mint.repository.EntityManagerWrapper.createEM(EntityManagerWrapper.java:50) ~[mint-1.170821.01-SNAPSHOT.jar:na]
at com.apigee.mint.repository.EntityManagerWrapper.<init>(EntityManagerWrapper.java:34) ~[mint-1.170821.01-SNAPSHOT.jar:na]
--
--
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590) [jetty-util-9.4.0.v20161208.jar:9.4.0.v20161208]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
2018-05-19 03:21:10,791  qtp2033268925-45 ERROR REST - CustomJAXRSInvoker.performInvocation() : CustomJAXRSInvoker.performInvocation : Method com.sun.proxy.$Proxy119.seedBillingDocumentNumbersForOrganization threw an exception.
com.apigee.kernel.exceptions.spi.UncheckedException: Unanticipated exception. Please contact your Support administrator with code : c4ea1427-a930-44bc-8287-53397bf128e6
at com.apigee.mint.exceptions.MintExceptionTranslator.translate(MintExceptionTranslator.java:101) ~[mint-1.170821.01-SNAPSHOT.jar:na]
at com.apigee.mint.exceptions.MintExceptionTranslator.translateToApiPlatform(MintExceptionTranslator.java:41) ~[mint-1.170821.01-SNAPSHOT.jar:na]
at com.apigee.mint.commons.AbstractExceptionHandlingCapableService.handleException(AbstractExceptionHandlingCapableService.java:79) ~[mint-configs-1.170821.01-SNAPSHOT.jar:na]
--
--
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672) [jetty-util-9.4.0.v20161208.jar:9.4.0.v20161208]
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590) [jetty-util-9.4.0.v20161208.jar:9.4.0.v20161208]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
2018-05-19 03:21:10,791  qtp2033268925-45 ERROR REST - ExceptionMapper.toResponse() : Error occurred : Unanticipated exception. Please contact your Support administrator with code : c4ea1427-a930-44bc-8287-53397bf128e6
com.apigee.mint.exceptions.MintExceptionTranslator.translate(MintExceptionTranslator.java:101)
com.apigee.mint.exceptions.MintExceptionTranslator.translateToApiPlatform(MintExceptionTranslator.java:41)
--
--
org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:173)
com.apigee.rest.framework.CustomJAXRSInvoker.performInvocation(CustomJAXRSInvoker.java:126)
2018-05-19 03:21:10,791  qtp2033268925-45 ERROR REST - ExceptionMapper.toResponse() : Returning error response : ErrorResponse{errorCode = mint.unhandledException, errorMessage = Unanticipated exception. Please contact your Support administrator with code : c4ea1427-a930-44bc-8287-53397bf128e6}
2018-05-19 03:24:01,001  Taurus_Worker-5 INFO  o.q.p.h.LoggingTriggerHistoryPlugin - LoggingTriggerHistoryPlugin.triggerFired() : Trigger DEFAULT.MINT.EMAIL_NOTIFICATION@@@management-server@@@MINT.EMAIL_NOTIFICATION@@@management-server@@@SYSTEM@@@management-server@@@SYSTEM fired job DEFAULT.MINT.EMAIL_NOTIFICATION@@@management-server@@@MINT.EMAIL_NOTIFICATION@@@management-server@@@SYSTEM@@@management-server@@@SYSTEM at:  03:24:01 05/19/2018
2018-05-19 03:24:01,003  Taurus_Worker-5 INFO  TAURUS.SCHEDULER.SERVICE - ZKLockManager.createLock() : Acquired lock /taurus/pods/central/locks/lock-mint.scheduler.notificationmp@@@management @ Sat May 19 03:24:01 UTC 2018 : status : true
2018-05-19 03:24:01,003  Taurus_Worker-5 INFO  o.q.p.h.LoggingJobHistoryPlugin - LoggingJobHistoryPlugin.jobToBeExecuted() : Job [DEFAULT.MINT.EMAIL_NOTIFICATION@@@management-server@@@MINT.EMAIL_NOTIFICATION@@@management-server@@@SYSTEM@@@management-server@@@SYSTEM] to be fired by trigger [DEFAULT.MINT.EMAIL_NOTIFICATION@@@management-server@@@MINT.EMAIL_NOTIFICATION@@@management-server@@@SYSTEM@@@management-server@@@SYSTEM], re-fire: 0

Report Inappropriate Content

I went through system log. I believe it is DB connection issue. Can you restart management server in debug mode and attach entire system log after restart here

kaushal

@njoshi

I am attaching the debug management server logs for your reference.

Please find below the database details :-

psql -h 127.0.0.1 -d apigee -U postgre

 rkms               | _version                       | table | postgre
 rkms               | address                        | table | postgre
 rkms               | application                    | table | postgre
 rkms               | application_category           | table | postgre
 rkms               | application_has_product        | table | postgre
 rkms               | bank_detail                    | table | postgre
 rkms               | billing_adjustment             | table | postgre
 rkms               | billing_doc_dev                | table | postgre
 rkms               | billing_doc_num_generator      | table | postgre
 rkms               | billing_doc_product            | table | postgre
 rkms               | billing_document               | table | postgre
 rkms               | charge_daily                   | table | postgre
 rkms               | charge_hourly                  | table | postgre
 rkms               | company                        | table | postgre
 rkms               | company_category               | table | postgre
 rkms               | currency_mapping               | table | postgre
 rkms               | custom_attribute               | table | postgre
 rkms               | dev_rate_plan_counter          | table | postgre
 rkms               | developer                      | table | postgre
 rkms               | developer_balance              | table | postgre
 rkms               | developer_balance_audit        | table | postgre
 rkms               | developer_category             | table | postgre
 rkms               | developer_has_rate_plan        | table | postgre
 rkms               | developer_has_role             | table | postgre
 rkms               | developer_role                 | table | postgre
 rkms               | developer_terms_and_conditions | table | postgre
 rkms               | history                        | table | postgre
 rkms               | job                            | table | postgre
 rkms               | limits                         | table | postgre
 rkms               | limits_counter                 | table | postgre
 rkms               | mint_locks                     | table | postgre
 rkms               | monetization_package           | table | postgre
 rkms               | notification_action            | table | postgre
 rkms               | notification_condition         | table | postgre
 rkms               | organization                   | table | postgre
 rkms               | package_has_product            | table | postgre
 rkms               | prod_dev_cat                   | table | postgre
 rkms               | product                        | table | postgre
 rkms               | product_broker                 | table | postgre
 rkms               | provider                       | table | postgre
 rkms               | rate_plan                      | table | postgre
 rkms               | rate_plan_details              | table | postgre
 rkms               | rate_plan_rates                | table | postgre
 rkms               | report_definition              | table | postgre
 rkms               | settings                       | table | postgre
 rkms               | supported_currency             | table | postgre
 rkms               | terms_and_conditions           | table | postgre
 rkms               | transaction                    | table | postgre
 rkms               | tx_brokerage                   | table | postgre
 rkms               | webhook                        | table | postgre

kaushal

@njoshi

I am attaching both the silent response configuration file for your reference to install Monetisation service and Monetization Onboarding. (https://docs.apigee.com/private-cloud/v4.18.01/installing-monetization-services)

Report Inappropriate Content

In "apix-mint.properties" change the config mint.openjpa.log and restart management server

mint.openjpa.log=File=/opt/apigee/var/log/edge-management-server/logs/openjpa.log, Query=ERROR, DefaultLevel=ERROR, Runtime=ERROR, Tool=ERROR

kaushal

[root@ip-172-31-30-164 conf]# /opt/apigee/apigee-service/bin/apigee-service  edge-management-server restart
apigee-service: edge-management-server: pid=2032
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: Not running (DEAD)
apigee-service: edge-management-server: OK
apigee-configutil: edge-management-server: # OK
The file conf/apix-mint.properties was changed
[ChangeDelta, position: 68, lines: [#mint.openjpa.log=File=/tmp/openjpa.log, Query=ERROR, DefaultLevel=ERROR, Runtime=ERROR, Tool=ERROR, mint.openjpa.log=File=/opt/apigee/var/log/edge-management-server/logs/openjpa.log, Query=ERROR, DefaultLevel=ERROR, Runtime=ERROR, Tool=ERROR] to [mint.openjpa.log=File=/tmp/openjpa.log, Query=ERROR, DefaultLevel=ERROR, Runtime=ERROR, Tool=ERROR]]
apigee-configutil: edge-mint-management-server: # OK
apigee-service: edge-management-server: Not running (NO_LOCKFILE)
apigee-service: edge-management-server: status=2, continuing
apigee-service: edge-management-server: OK
apigee-service: edge-management-server: edge-management-server is running
[root@ip-172-31-30-164 conf]#

When i restart it gets reset to /tmp/openjpa.log

Report Inappropriate Content

Change property "conf_apix-mint_mint.openjpa.log" in /opt/apigee/customer/application/mint-management-server.properties

conf_apix-mint_mint.openjpa.log=File=/opt/apigee/var/log/edge-management-server/logs/openjpa.log, Query=ERROR, DefaultLevel=ERROR, Runtime=ERROR, Tool=ERROR

hariprasadar

Is that above issue resolved?

oshum

Was the issue resolved? Could you please share how? I have similar issue, don't know where to start.

mint.unhandledException while enabling monetization service for an organisation