From NewHaven Software Wiki

Current revision as of 14:25, 23 November 2020

Overview

CMS uses XML as its primary means of order importing and also is used for assorted other data imports and exports. A full specification is defined in our document CMSXML.xsd. This document is available on request from NewHaven Software with a signed non-disclosure agreement.

For more information on order file importing and eCommerce integration capabilities please see our related article.

The XML

XML is our file format of choice. It is extremely flexible, robust, and widely used. If you are unable to obtain your orders in our described XML format, NewHaven Software can be contracted to write a transform for you that will convert your current file format (ASCII, other XML, etc.) into our XML for importing into CMS. This can be done with a few clicks or, with our eCMS module, fully automated.

The balance of this guide provides additional information about the preparation and use of XML with CMS.

Transforms

It is unlikely CMS will support other specifications/formats in the future. Instead, whenever we need to import from a source that cannot provide XML in our format, we would instead put that data through a transform process to convert it from their native format into CMS's expected XML format.

Transforms like this are currently done with our eCommerce integrations: CommerveV3, Magento, and Volusion, for example. When importing a file from one of those sources, the CMS plug-in for that store calls a .dll to transform the file on the fly into our spec and then the file is imported. This process is transparent to the user. Something similar is possible using a .xslt transform in conjunction with our generic plug-in.

Another solution is to use a tool called MapForce that can be purchased (not from us) and used by our clients if they wish to write/execute their own transforms. This includes converting not only from other XML formats but also from ASCII, spreadsheets, etc.

Your first choice, however, should be to work with the company providing your data and establish if they can do the transform on their side.

XML Standards

There are two distinct phases of ensuring that the XML you produce will work with the merchant's system.

1. Ensure your XML is valid (covered by this section) - Your XML must be valid, both per XML standards and meeting the requirement of our XSD specification
2. Address the merchant's specific data requirements

Data Validation

The CMSXML.xsd file we supply outlines the specific needs of our system including structure of the XML, required elements and the sequence they appear in, definitions of the types of data allowed in an element (length, numeric, etc.).

Data validation should be employed on your site to ensure customers are entering data suitable for that field (e.g. numbers in a phone field.)

Handling Special Characters

There are also generic XML standards not explicitly expressed in the XSD that should be followed. A common issue is the use of special characters as data that are reserved for XML encoding. Your system should be able to find and properly escape (encode) these characters to ensure your XML file can be properly parsed. This wiki is a good resource for the list of characters:

https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references#Predefined_entities_in_XML

Example:

Salt & Pepper should instead be passed as Salt & Pepper.

Validating Your XML

Before submitting your XML for testing, please validate it against our CMSXML.xsd (provided on request with NDA). If you do not have your own tool for doing so, there are free online tools available that allow you to load your file and our XSD to show you where there are problems with the structure of the file.

Merchant Data Requirements

The information listed here addresses common questions and mistakes made when preparing XML for importing orders and/or catalog requests into CMS.

Tech Tips

• Orders imported are treated as new orders. You cannot use XML to import order history.
• Imported catalog requests will not update a customer's address if a customer number is being passed to match it to an existing customer.
• Matching
  • CMS will attempt to match imported orders and catalog requests to existing customers based on this logic when a customer number is not passed.
  • New customer records will be created if no match can be established.
• All orders must be manually verified before saved as new orders or use the optional Automated Imports Module to validate as many as can pass your chosen validation checks.
• Inventory for imported orders is allocated at the time of validating.
• CMS order numbers are assigned to imported orders at the time of saving the validated order, not when imported. CMS will retain the order number generated by your site (or source of the order) in a field called Reference Order ID which can be searched on, printed, exported with orders, or even passed through some shipping integrations to appear on shipping labels.
• Catalog requests do not require nor can they be manually verified. They are automatically processed.
• Catalog requests and orders both require unique identification numbers (ID and Reforderid respectively)

Intro

Every effort should be made to ensure the product codes, taxing, and shipping & handling charges are setup to match what is in CMS (or that CMS is setup to match the site's config). CMS will expect your XML to contain all financial elements of the order including product pricing, discounts, shipping and handling, insurance, and payment. While we also encourage you to include tax in your XML, this is the only charge the CMS will override during import. In other words, your XML will dictate all of the financials of the order and CMS will not override or recalculate any of your financials with the exception of tax. Tax will be calculated as orders are imported. As such, you should plan on including all pertinent financials in your XML and configure your system to calculate tax in the same way as your client as configured CMS to calculate tax.

Products

The most important and potentially challenging aspect of website integration is ensuring the products are identified the same way in both systems. If both are setup at the same time and have the same capabilities this may not be so difficult but often they are disparate systems that were set up independent of each other and potentially with different product options.

In most situations the goal is to facilitate not only order downloads from the site into CMS but also product data uploads from CMS to the site which would include new products, price changes, current stock availability, etc. In other words, it's critical that both CMS and the site can identify products in the same way.

Products are uniquely identified by CMS's product code in most integrations. Where applicable there may also be a size/color type attribute. In more advanced integrations like we have with Magento, it is also possible to use an underlying SKU_ID to match up products that do not share the same product code or structure (e.g. your site does not support size/color attributes but you use them in CMS.)

Determining your strategy for matching products between the two systems should be your first order of business. If any changes are required, and you have a lot of products, this can be a time consuming task that you don't want to put off and risk launch delays or unexpected results.

Shipping Methods

For the shipping methods we recommend you use the 'hotkeys' the user has setup for each shipping method in CMS and pass it in the XML as such:

<DefaultShipMethod IndexBy="Code">FG</DefaultShipMethod>

where FG is the hotkey they've setup in CMS to represent FedEx Ground, for example.

The above corresponds to the order's default shipping method and will be used for all packages on the order.

You can additionally pass a shipping method at the recipient/package level if a recipient on the order must ship via an method other than the order default. We recommend only setting this when they differ since CMS treats this explicit setting of a recipient method as an override that cannot be automatically updated by any changes the user makes to the order's default shipping method (an edit that would otherwise cascade to alter the shipping methods on all recipients not explicitly set.)

Addresses

At minimum an address should be passed in the <Customer> node (this is the primary address) and the <Recipient> node (this is the shipping address). The billing address is optional and will default to the address passed in the Customer node if a billing address is not passed. In CMS, AddressLine3 is the primary address line. Any additional address lines should go onto AddressLine2 and only use AddressLine2 if 3 and 2 are already used.

Notes

<Notes> should be used for notes to the buyer. <GiftMessage> should be used for notes to a gift recipient (when the recipient is not the buyer.)

Countries

If you do not pass a country CMS will assume the country is the same as that defined in CMS Setup for the company's address. If you are passing the country you should use <CountryID> and use ISO3166-1 codes. CMS will recognize alpha-2, alpha-3, or the numeric codes (3 digit).

CountryName can be used but requires an EXACT match with the ISO3166 or UPS list of country names (names, not the 2 digit code) which has a much lower chance of matching correctly, particularly if you allow free form entry of country name instead of choosing from a list.

Payment Methods

Payment methods they must be passed in the XML using the following names: (note - XML is case sensitive)

• American Express
• Check
• JCB
• MasterCard
• Diner's Club
• Money Order
• Discover
• Visa
• PayPal
• GoogleCheckout

Any user-defined payment methods created in CMS that are to be used on the website must also have their names match identically, including letter case.

Encryption

CMS does not currently support a method of field or file level encryption with XML. You may consider using PGP as a solution for encryption and then decrypt just before importing into CMS.

When using our eCMS Module to connect to your web site's web service or an FTP location, CMS can enforce the use of secure connections (TLS) for orders that contain credit card data. This method ensures that a file with unencrypted card data never resides on your machine/network. The order data goes straight into the database (never saved as a file) and is encrypted before it is written to the table.

CMS is a PA-DSS validated application (2010 - present, v7 - 10.x) and you can rest assured that the credit card data will be managed in a PCI compliant manner once imported into CMS.

Credit Card Processing

Our XML supports importing unapproved, approved or even captured credit cards. We strongly discourage capturing credit cards outside of CMS, however. On import, CMS can adjust the payment amount to match the new amount of the order if the order total needs to be adjusted down due to items on backorder, for example. It can also account for small variations in tax amounts (as can happen if your site cannot calc tax the same way CMS can.)

If you'll be authorizing on your site and capturing in CMS (recommended) you'll need to use the same payment gateway and merchant account on both sides. You'll also need to pass additional data fields like a transaction_id to ensure CMS is able to capture the payment. Contact us for more details specific to your processor (Authorize.net, MPS/Vantiv, TransFirst).

Also, make sure your expiration dates are in the proper format of YYYY-MM. Improper formatting of the expiration date is probably the most common error we see.

Testing

Data Provider

1. We can provide a sample xml file with orders that reflect a number of common product and order scenarios. You should discuss specific needs with the client to see which scenarios you should prepare for (e.g. multiple recipient orders, catalog requests, future shipping orders, credit card payments, etc.)
2. As stated above, please ensure your XML validates against the provided XSD. CMS is tolerant of nodes and elements in the wrong sequence, which would normally raise errors when validating, but best to get those right just the same.

Client

We can assist you with setting up a test database to facilitate testing of imports before they are used in production. When testing, you should both be testing for:

• the success of the import (no errors)
• ensure all required data is being received in the right places