Third Party Feeds: Network Q/Datafirst Integration

5 minute read

Network Q is the used car brand of Vauxhall that offers the consumer piece of mind when buying a used car. Vauxhall, a UK based marque owned by General Motors (GM) allows Vauxhall dealers to list their Network Q qualifying vehicles on their website. In my previous post: Third Party Stock/Inventory Data Feeds I talked about how important and how much choice there is now when displaying your vehicles online, with the Network Q website, its free for dealers to list their vehicles. This integration saves the dealer manually uploading the vehicles one by one though the admin area and can be fully automated by taking the stock from their DMS (Dealer Management System) such as Kerridge/ADP, Pinnacle etc.

The vehicles need to undergo a set of multi point checks and meet some other requirements such as age and mileage.

Vauxhall use a company called Datafirst who are based in France to manage the Network Q website. Datafirst offer a webservice API (Application Programming Interface) for you to upload the stock. Unlike most third party websites that accept feeds the API the Datafirst provide allows the updates to be made in real time. From a developer’s perspective this is far better to work with than CSV (Comma Separated Values) file. The biggest problem with CSV files is that they normally get transferred via FTP (File Transfer Protocol) which makes it difficult to determine whether the transfer succeeded or not.

The Datafirst API is session based and a number of calls need to be made in a particular order.

The main decision you need to make is how the webservice gets called. Normally I would have built an SSIS (SQL Server Integration Services) package to deal with this by utilising the webservice and xml tasks. But due to a number of reasons, such as deployment etc. I opted for a simple console application that is run from Windows scheduled task manager.

The 4 steps required are:

Initialise Session
Send Vehicle Data
Send Vehicle Photos
Finalise Session

Something I recommend is that you put good error checking at every stage plus a catch all to catch unhandled errors. Is not uncommon to find that Datafirst’s service is unavailable from time to time. The error checking will alert you by email so you can rerun or notify someone of the failure.

I do this by adding the following piece of code:

AppDomain currentDomain = AppDomain.CurrentDomain;

currentDomain.UnhandledException += new UnhandledExceptionEventHandler(UnhandledExceptionHandler);

  

static void UnhandledExceptionHandler(object sender, UnhandledExceptionEventArgs args)

        {

            StringBuilder sb = new StringBuilder();

            

            Exception ex = ((Exception)(args.ExceptionObject));

            sb.Append("****** Unhandled Exception ******<br/><br/>");

            sb.Append("<br/>ExceptionType: " + ex.GetType().FullName);

            sb.Append("<br/>HelpLine: " + ex.HelpLink);

            sb.Append("<br/>Message: " + ex.Message);

            sb.Append("<br/>Source: " + ex.Source);

            sb.Append("<br/>StackTrace: " + ex.StackTrace);

            sb.Append("<br/>TargetSite: " + ex.TargetSite);

  

            Exception ie = ex;

            while (!((ex.InnerException == null)))

            {

                ie = ie.InnerException;

                sb.Append("<br/><br/>****** Inner Exception ******<br/><br/>");

                sb.Append("<br/>ExceptionType: " + ie.GetType().Name);

                sb.Append("<br/>HelpLine: " + ie.HelpLink);

                sb.Append("<br/>Message: " + ie.Message);

                sb.Append("<br/>Source: " + ie.Source);

                sb.Append("<br/>StackTrace: " + ie.StackTrace);

                sb.Append("<br/>TargetSite: " + ie.TargetSite);

            }

  

            Utilities.SendEmail(Utilities.Preferences.ErrorEmails, sb.ToString());

        }

The next thing to do is set up the objects we need to pass over. First off we create a base class with only one method that serialises the object.

public abstract class RequestBase

    {

        public string SerialiseToString()

        {

            XmlSerializer xs = new XmlSerializer(this.GetType());

            XmlWriterSettings xws = new XmlWriterSettings();

            xws.Encoding = new UTF8Encoding(false);

            string xml = string.Empty;

  

            using (MemoryStream ms = new MemoryStream())

            {

                using (XmlWriter xw = XmlWriter.Create(ms, xws))

                {

                    xs.Serialize(xw, this);

                    xw.Close();

                }

                xml = Encoding.UTF8.GetString(ms.ToArray());

            }

  

            return xml;

        }

    }

Then the objects, here is the vehicle request object that takes a collection of vehicles.

[Serializable]

[XmlRoot("REQUEST")]

public class VehicleRequest : RequestBase

{

    [XmlAttribute("NAME")]

    public string NAME { get; set; }

    [XmlAttribute("SESSID")]

    public string SESSID { get; set; }

  

    [XmlArrayAttribute("UVSTOCK")]

    [XmlArrayItemAttribute("USEDVEH")]

    public List<Vehicle> Vehicles { get; set; }  

}

Once all the objects are specified and marked-up with the relevant attributes we need to send them over to the webservice. Now, normally it would be easy to create a proxy class and call the webservice direct from code, but the way that Datafirst work is just to receive POX (Plain Old Xml) via HTTP requests and responses. To send the data then we need to us the HttpWebRequest from the System.Net namespace like so:

public static string PostRequest(string xml, string uri)

{

    Uri address = new Uri(uri);

    string responseXml = string.Empty;

  

    HttpWebRequest request = WebRequest.Create(address) as HttpWebRequest;

    request.Method = "POST";

    request.ContentType = "text/xml";

  

    byte[] byteData = UTF8Encoding.UTF8.GetBytes(xml.ToString());

  

    request.ContentLength = byteData.Length;

 

    using (Stream postStream = request.GetRequestStream())

    {

        postStream.Write(byteData, 0, byteData.Length);

    }

  

    // Get response   

    using (HttpWebResponse response = request.GetResponse() as HttpWebResponse)

    {

        StreamReader reader = new StreamReader(response.GetResponseStream());

        responseXml = reader.ReadToEnd();

    }

    UpdateSession(responseXml);

  

    return responseXml;

}

Now you might be wondering how do the images get transmitted. Well, we need to encode the images in base64.

internal static string ConvertImageToBase64(string path)

{

    byte[] byteArray = null;

  

    using(FileStream fs = new FileStream(path, FileMode.Open, FileAccess.Read))

    {

        byteArray = new byte[fs.Length];

        fs.Position = 0;

        fs.Read(byteArray, 0, (int)fs.Length);

    }

  

    return Convert.ToBase64String(byteArray);

}

It’s recommended due to the amount of vehicle images that can be transferred to only send over modified or new images, so you’ll need to update the database with a sent flag or similar. Alternatively, what I did that works well is to set the creation time of image to some time in the past and the next time you look for the images only get images without that date. This is extremely useful if this application was to be deployed to a computer within a dealership or you don’t want to mess around with databases.

So there you have it. Not difficult but different to how other people do it, and in my opinion a lot better then the CSV/FTP model I tend to come across.

Share on

Twitter Facebook Google+ LinkedIn

Steve Taylor

Share on