How do I move data from one org to another ?

[#1]

From time to time we get a question from users: how can I move data from one Edge organization into another? Or, how can I backup my configuration so that I could restore it at a later point?

For this purpose we have created the "Apigee Organization Data Migration Tool"! Find it on github:

https://github.com/apigeecs/apigee-migrate-tool

Typical Use Cases :

1. Move data from one organization to another. For example move data from your free org to a paid org.
2. Import developers, apps and keys from a csv file to an Apigee organization.
3. Export proxies, developers, products, apps and keys for backup.

Features

1. Export Data from an organization
   • Developers
   • Proxies (latest version)
   • Products
   • Apps
   • App Keys
2. Import Data to an organization
   • Developers
   • Proxies (latest version) and deploy them
   • Products
   • Apps
   • App Keys
3. Import Data from csv file to an organization
   • Developers
   • Apps
   • App Keys

For installation and usage instructions please visit - https://github.com/apigeecs/apigee-migrate-tool

And please do provide feedback on the tool via github. We accept pull requests and issue reports of course!

[divyaachan]

@shah do you know if this tool is officially supported. If a customer rans into any issues using this tool, whom do they reach to?

[#3]

@Divya Achan - This tool is not officially supported by Apigee. It is an open source tool under MIT license terms. Customer is free to use it at risk.

[#4]

Thank you @shah for writing awesome tool.

This is big help to get or transfer data from one org to another org.

Only one question : How do I pass my both org URL in config.js ?

I did 'https://api.enterprise.apigee.com/v1/organizations/{org-name}'

But, it's give me an error. I am wrong somewhere in url but, don't know where? Could please help me out here.

Thank you.

[#5]

The org-name would go in the org field in the corresponding from and to section. The url would be the same if both orgs are in Apigee Edge cloud.

The config.js would look like below

from: { version: '14.01', url: 'https://api.enterprise.apigee.com', userid: '{userid}', passwd: '{password}', org: '{org1}', env: 'test' },

to: { version: '15.01', url: 'https://api.enterprise.apigee.com', userid: '{userid}', passwd: '{password}', org: '{org2}', env: 'test' }

[#6]

Thank you @shah

But, my config.js is like this:

module.exports = { from:

{ version: '1',

url: 'https://api.enterprise.apigee.com’,

userid: 'userid’,

passwd: 'pass',

org: 'dhwani-prod',

env: 'dev' },

to: {

version: '1',

url: 'https://api.enterprise.apigee.com’,

userid: 'userid’,

passwd: 'pass',

org: 'dhwani',

env: 'dev' }

} ;

I am getting error on my url: "Unexpected token ILLEGAL"

Do you think instead of http protocol , I am using https so, that is causing trouble?

[#7]

No. Definitely works with https. It is not able to parse the url from config.js. I pasted your config in an editor. The end quote of the url field is bad 🙂

[adas]

@shah

This definitely looks like a promising tool. Based on what I see above, I had the following questions:

- Does this tool allow migration of environments, virtualhosts, keystores etc ?

- Does the tool have support for migrating org/env level resources like resource files, cache resources, target servers etc ?

Apart from developers, apps, keys and proxies, these are major resources which a customer would like to migrate from their free org to paid orgs, or non-prod to prod orgs.

[#9]

Arghya - This is currently not supported and something I plan to add to the tool. Also pull requests are welcome 🙂

[#10]

Thank you @shah . I have modified scripts a bit as something is change in Apigee edge and it's not supported. But, now, I temporarily fix it and it is exporting all data and also importing too.

Thank you for this wonderful tool. 🙂

[akashtp]

@shah thanks for the wonderful article.

Is there a way by which OPDK users can move data into a new org? I understand that one will have to create a new org and migrate data into that.

Is there a set of procedures/guidelines supported by Apigee in order to accomplish this?

[#12]

The tool uses management API's and works for both opdk and cloud deployments. The opdk installation guide covers how to create orgs, env etc. There is a script setup-org.sh with the opdk, which can be used to create a new org, environment, roles, users etc.

[#13]

let me start with thanking you @shah for creating this tool. This has helped me a lot to get my data from the ancient version of Apigee planet to 15.01 Apigee planet.

Now here comes the question:

• Is there a way to migrate data from one ORG to another ORG based on following conditions:
  • different planets and different ORG names ?
  • different planets with same ORG name but different environments ?
    • I had a requirement for this. The only way I could hack it by creating a same environment on destination planet [just for the import purpose]. Once the data was imported I deployed to the them to other environments on new planet. I might delete the the dummy environment now as I have no need to for it.
      • Please let me know if you have any other way to achieve this.

[#14]

Hi Satyajit,

See my responses below ..

Is there a way to migrate data from one ORG to another ORG based on following conditions:

• different planets and different ORG names ? Yes. Set the url for the management server (could be on different planets) and the org name in the config.js file.
• different planets with same ORG name but different environments ? The env to be imported is also specified in the config.js file. Make sure that the environment specified is pre-created in your target planet. Note the env configuration is currently only used for proxy deployment

[#15]

So you mean that below is possible:

module.exports = {

    from: {
        version: 'r23',
        url: 'http://Source-MGMT-IP:PORT',
        userid: 'Username',
        passwd: 'Password',
        org: 'ORG',
        env: 'SOURCE-ENV'
    },
    to: {
        version: '15.01',
        url: 'http://Destination-MGMT-IP:PORT',
        userid: 'Username',
        passwd: 'Password',
        org: 'ORG',
        env: 'DESTINATION-ENV'
    }
} ;

As long as I have "DESTINATION-ENV" already created. This should work...

[#16]

Yes. It should work.

[#17]

@shah quick question about the migration. May be it is a dumb question but I am fairly new to Aigee. Does proxy police's order matters ? After I moved data from my old planet to new planet. I have been seeing that the ordering for the policy is changed. Also " are not ". Old instances use to show double quotes as """.

[#18]

By "Does proxy police's order matters ?" - you mean the order of policies used within a proxy bundle ? It could matter depending on what you do in your proxy.

The tool does not update the proxy bundle it just exports and imports the bundle as it is. So the order of polices within the bundle are not changed by the migration tool.

[#19]

@shah quick question, in process of building new prod system. I have to do clean up multiple times. But even if I delete everything [means delete VH, ENV and ORG as well], and re-run

setup-org.sh

to create new ORG, ENV and VH. I get this error

--> curl -X POST http://{MGMT}:{PORT}/v1/organizations/{ORG}/environments/prod/analytics/admin?type=organizations [...]
{
  "code" : "Scope ORG~prod already registered in group axgroup001",
  "message" : "Scope ORG~prod already registered in group axgroup001",
  "contexts" : [ ]
}
==> 400

ERROR: Analytics onboard for ORG/prod failed.
Continue anyhow y/(n) y

I understand that it is complaining about the analytics POD. What steps are required to fix this and how do I make sure that it is working.

[jineshnarenthak]

Hi Shah,

Following errors are observed while migrating the keys,can you please help

>> ERROR Resp [500]

>> "code" : "keymanagement.service.invalid_apiproduct_for_key",

>> "message" : "ApiProduct named <API Product Name> is not associated with the given key",

Thanks for the help.

Regards

[#21]

you can check this post maybe its help you its all about ip address and i solved my problem of th same " restrictions do I face when using 192.168.2.1 "