Data Management Part I: What Are Data Management Platforms?
Data Management Part II: Centralize and Synchronize Your User Data
Data Management Part III: Syncing Online Data to a Data Management Platform
Data Management Part IV: Syncing Offline Data To Your DMP

I’d thoroughly recommend reading these as they are a great introduction to the whole data/DMP field from a specifically AdOps perspective. Irrespective of whether you currently have any plans or strategies around leveraging the data your various adserving, analytics & behavioural platforms provide, these articles will give a solid view of what and how these technologies can start to work together.

If you’ve not seen this already Joelle Frijters over at Improve Digital has recently posted up an article which takes that Luma landscape concept and reworks it.

Personally I prefer the Improve Digital version. Not only does it simplify the layout but it logically groups them into their core areas of specialism (and in doing so I feel clarifies the current field or providers & operators) but there’s also a helpful glossary of the every-changing acronyms we like to use.

Article: http://www.improvedigital.com/improve-digitals-display-advertising-eco-system-map-201112-gets-great-reviews
PDF: here

On the desktop I have access to numerous tools that help me understand and debug what my adserver is doing. Tools such as httpWatch, httpFox, Fiddler2, Charles, HTTPScoop and FireBug. These tools intercept HTTP traffic and allow you to analyse, check and view the response from the remote adserver. I use this extensively to test response times, debug connectivity & delivery issues, view mimetypes & cookie data amongst other tasks. Once you’ve used these tools it’s difficult to understand how you’ve managed so long without them… that is until you start delivering ads outside of your desktop environment where these tools no longer exists or the devices are unable to provide this information natively or without jailbreaking/rooting your phone.

So, when we started developing the next iteration of our iPhone app we needed to create an environment where we could replicate the tools we’ve come to expect on the desktop for the mobile app as well. Having just moved our entire team to MacOS from Windows based machines this proved to be interesting.

How this works:
Basically we are creating an ad-hoc network on your Mac desktop which the iPhone or other device can connect to. We are then going to route the mobile traffic through your desktop’s Ethernet connection. HTTPScoop sits on the ethernet connection watching and reporting all traffic that flows through it to port 80 (or whatever port you have decided to monitor). This method allows you to visualise the HTTP traffic of your iPhone/Android etc on your desktop and enable you to check these calls are being made correctly. This tutorial describes how to setup your desktop (I’ve used Snow Leopard, MacOS X here) to intercept and display HTTP requests using HTTPScoop made by your mobile device. it requires that you have or are running the following:

Requirements:

• MacOS X (I’m using 10.6 – Snow Leopard) running Airport
• A working copy of HTTPScoop (http://www.tuffcode.com/) Free 14 day trial or only £10 per licence
• An active internet connection.
• Mobile device running the app you need to test

Process Outline: This is what we’ll be covering.

1. Using Internet Sharing create an ad-hoc wireless network through your Airport card.
2. Connect mobile device to the ad-hoc wireless network
3. Start HTTPScoop and configure to show http ethernet traffic
4. Activate the mobile app and watch the results.

Create an ad-hoc wireless network through your Airport card.

1. Got to System Preferences and click Sharing under the ‘Internet & Wirleess‘ group.
2. If needed click the padlock to make changes
3. In the Services list click Web Sharing, we need to activate this as otherwise internet sharing will not work properly for some reason.
4. Next click the Internet Sharing service, options (‘Share your connections from’ and ‘To computers using’) will appear to the right of this.
5. Choose Ethernet from ‘Share your connection from:‘
• tick Airport from the list ‘To computers using‘
• click ‘AirPort Options‘
• Name your Network eg; ‘AdOps’
• Enable WEP encryption and add suitable password if needed (it is WEP after all!) & click OK
• Tick the Internet Sharing box (in the Services list) to activate it, this will grey out the Internet Sharing options., the icon should now have turned green (previously it may have been red). You will be asked for your password to authrorise this.
6. Tick the Internet Sharing Box this will activate Internet Sharing, you will be asked to confirm this is what you want, click ‘Start‘ to continue.
7. Retick the padlock to submit and lock the changes.
8. Your ad-hoc wireless is now setup and ready for you to connect your mobile device to it.

Start HTTPScoop and configure to show http traffic: Connecting to and tracing/view those HTTP calls.

1. Start your copy of HTTPScoop. Next we will configure it to show the traffic you want.
2. Choose the Ethernet (‘eth0‘ or ‘en0‘) option from the dropdown to the right of the HTTPScoop window) then click ‘Scoop‘ – this will start to track all HTTP traffic on your ethernet connection NB: You may see other traffic appear if you have other apps/browser running in the background which are active. To make things simpler close any browsers and other apps using HTTP Traffic down (eg; RSS readers).
3. Click the ‘Scoop’ button, this will start recording the HTTP traffic, as this is a privileged service you will need to enter your password.

Connect mobile device to the ad-hoc wireless network:

1. Connect your iPhone/Android/Tablet or other device to the newly created ad-hoc network.
2. Start the application you wish to monitor, after a while & when the app is making HTTP calls you will start to see data being recorded into HTTPscoop.

Caveats & Postscript:

• HTTPScoop by default only monitors HTTP traffic on port 80. If you need to monitor other traffic eg; HTTPS on port 443 then this can be accomplished by turning HTTPScoop off and amending the ports monitored through HTTPScoop->Preferences.
• Yes, I am aware that this method does theoretically allow you to set up and adhoc network and monitor any attached device (whether it belongs to you or you have permission to view this data or not) so please use this method wisely. Be aware that you MUST have permission to intercept the mobile device’s HTTP Traffic, capturing unauthorised HTTP traffic may be illegal in some jurisdictions..
• HTTPScoop has more help/information here: http://www.tuffcode.com/support.html
• I’ll be posting alternative methods using different software (eg; Charles debugging Proxy) as well as methods that work on Windows very soon.

Links:
HTTPScoop

In the past we used, and still do, tools such as Sothink SWF Decompiler and URLActionEditor to analyse flash files and amend them accordingly. But enter OneVision wo’ve been busy in the intervening years and by all accounts have built out a fully-featured pre-flighting tool called Contenio for use by online AdOps departments to check and authorise creatives as they come into the department. I haven’t yet seen the product working but I’m hopeful it will address some of the ore recurrent themes such as checking for clickTag() case sensitivity/spelling, fix _root. issues, check for embedded urls, check for target=”_blank” etc, but hopefully we’ll get to see this in action soon and report back our findings.

For those of you trying to automate out your QA process this could be of vital interest. I look forward to speaking to someone who’s currently or planning on working with this soon.

Enter Spoon.net a recent discovery which has managed to resolve this problem. Thanks to their virtualisation software I have been able to have IE6, IE7 and IE8 all running at the same time whilst all using httpWatch and AOL’s WebPageTest concurrently as well.

It’s not just browsers either, here’s a blurb from their site:
Spoon specializes in app virtualization and delivery technology, a family of sophisticated techniques that decouple apps from the underlying operating system and enable extremely rapid delivery and launch. In support of its ongoing product development efforts, Spoon conducts fundamental research in the areas of virtualization, operating systems, formal semantics, programming languages, complexity theory, and cryptography.

You’ll need to install Spoon.net’s plugin first but this is no big deal. Once installed simply click on one of the icons of the browser you want to test, if this is your first time launching that browser version it will download what I can only assume is the a local ‘image’ of that browser and when this is down will launch it just like any other application.

All in all this is an excellent site and well worth adding to your AdOps toolbox.

Downsides: Not supported on the Mac or on Chrome or on Opera from my initial tests.

Recently I’ve been looking at ways we can programmatically analyse ad file types and then test them against agreed standards. This will be for GIFs, JPG, PNG and more importantly SWFs. However, SWF decompilation can be a tricky beast and I plan on delving into the more in-depth in a related article but for the meantime here are a few tools that can accomplish it that we use ourselves. So lets begin:

SWFTools: [http://www.swftools.org/]
SWFTools is a set of command-line utilities for interacting with Flash files on the server. They provide the ability to create and decompile flash units and as they are on the command line means that web-services are able to interact with them, meaning you can build your own web-tools or incorporate them into back-systems you may have built in-house. I’m not going to delve into how to use this tool here just yet but I will go about showing you how, given shell access to your hosting account you should be able to install these tools locally.

The quick answer to getting this up on your account is to amend the ./configure line to include a –prefix=/home/UASERNAMEHERE/swtoolsinstalldir I’ll cover this in full below.

The SWFTools FAQ details standard installation as:

tar -zvxf swftools-0.x.x.tar
cd swftools-0.x.x
./configure
make
make install

This would fail on Dreamhost accounts as the make install requires root access which you’re not going to get top be honest. But never fear, this is easily circumvented without putting security at risk. The key here is to not try and install this on a machine-wide basis, that is create your own instance this program locally to your account.

• Create a directory called swftools in your home directory
• chdir into this directory and download the required tar.gz [At time of writing this was http://www.swftools.org/swftools-2009-08-24-2042.tar.gz]
• untar/unzip this to that dir
• chdir to the the folder this has created
• run the nromal installation script but change the ./configure line to “./configure –prefix=/home/USERNAMEHERE/swftools/” where USERNAMEHERE is replaced by your username
• complete the rest of script as detailed in the FAQ
• after this is installed change up a directory and remove the source files and directories

I’ve written this out a single list of commands here, remember to change the USERNAMEHERE to your account username.

cd ~/
mkdir swftools
cd swftools
wget http://www.swftools.org/swftools-2009-08-24-2042.tar.gz
tar -zxvf swftools-2009-08-24-2042.tar.gz
cd swftools-2009-08-24-2042
./configure --prefix=/home/USERNAME/swftools
make
make install
cd ..
rm -rf swftools-*

And that should be that To verify your installation do the following:

~/swftools/bin/swfdump -V

You should get back a response along the lines of

swfdump - part of swftools 2009-08-24-2042

If that’s there then Bob is your proverbial uncle and you’ve successfully managed to install SWFTools, well done.

NOTES:
When I configured my SWFTools I got one error:

checking for missing libraries... ungif gif_lib.h
***************************************************
* The following headers/libraries are missing: ungif gif_lib.h
* Disabling gif2swf tool...
***************************************************

Which means I don’t have access to the gif2swf feature, but that was not needed for my purposes. When I work out how to fix this I’ll post this fix back here as well. Have fun and please post back your successes and any interesting tolls you’ve created from it… happy coding.