December 18th, 2013 | Tags:

rubicon-project-logoIf like us you use Rubicon Project you’ll have noticed that each adunit is suffixed by a -N where N is an integer that indicates the size of the ad being requested. As we only only use 3 ad-sizes I’d never seen other rvalues than this, but was recently playing with their new RTP system and noticed the following snuck away in their ‘dorothy.js’ code. I think the interesting thing here is the surprising amount of different adsizes that are being traded programmatically outside of the IAB standards.

Interestingly there seems to be two elements that indicate multiple sized (width) units ’120/160′ and ’468/728′. So, if like ourselves you’ve written your Rubicon Tags dynamically then this may be of use to you.


this.ad_sizes = {
"120/160": "22",
"120x240": "12",
"120x60" : "6",
"120x600": "8",
"120x90" : "5",
"125x125": "7",
"160x600": "9",
"160x90" : "24",
"180x150": "11",
"180x90" : "25",
"1x1" : "30",
"200x200": "13",
"200x90" : "26",
"234x60" : "3",
"240x400": "17",
"250x250": "14",
"300x100": "19",
"300x250": "15",
"300x600": "10",
"300x850": "29",
"336x280": "16",
"468/728": "21",
"468x15" : "27",
"468x60" : "1",
"728x15" : "28",
"728x90" : "2",
"88x31" : "4",
"Pop" : "20"
};

November 21st, 2013 | Tags:

A couple of things bugged me about trying to import data from my DFP Impression logs into a simple MySQL database. Mainly it was down to Google using some very obscure character to delimit the text. In our case (and I believe you can ask them to change this) they are using the ‘thorn’ character – þ

So, my initial thought was to do a simple stream search/replace using sed and then import the file in locally to MySQL. The only thing I was worried about was the size of the logs I was working with, a single day’s NetworkImpression data was averaging 25GB. This works out at >1GB per hour of logs!

So, here’s as simple script that I use to unzip the file, convert the þ to a \t (tab) and then import the data into my database. Your log files will be named something like:
NetworkImpressions_NETWORKID_YYYYMMDD_HH.gz
eg;
NetworkImpressions_123456_20131114_08.gz

The script requires you just use the name of the file (without the .gz fielextension). I called my script ‘importdfp’, so to import a file called NetworkImpressions_123456_20131114_08.gz I would invoke the script:

./importdfp NetworkImpressions_123456_20131114_08

This is rough and ready but get’s the job done, I’m currently working on adding timing and logging to give a bit more feedback, as the DB table will reindex any new rows as it’s bringing these in it can be a little time consuming….

#unzip the file
gunzip $1.gz
#convert the thorn to a tab to a temp file, then delete the original and rename the file to the original filename
sed s/\\xFE/\\t/g $1 > $1_TMP
rm $1
mv $1_TMP $1
# import into MySQL
mysql -u USERNAME -pPASSWORD -e ‘LOAD DATA INFILE “/home/location/of/your/dfp/log/files/’$1′” INTO TABLE `LOG_IMPS` FIELDS TERMINATED BY “\t” ESCAPED BY “\\” LINES TERMINATED BY “\n” IGNORE 1 LINES;’ DFP

Yes, I know I should be using something like Hadoop, but this is a rough-and-ready test to prove we could bring the data in in a simple fashion, query it and get some basic data out. I was doing this on a simple dual VPS with 2GB of memory and it was taking approx 5-7mins to import each log file (including unzipping, sed transformations and loading the file into the DB and indexing the records).

November 14th, 2013 | Tags: , , , ,

So, a very long time since I posted here, shows you how busy this industry is at times I suppose. So, recently I’ve been working with the copious amount of logs that DFP produces for us, specifically I’m trying to look at the custom variables we pass as I’m trying to analyse mobile traffic a little better, Mobile devices are a little more temperamental than desktops and so all the data/key-value-pairs we try and put into our tags may not always propagate through to the tags.

So in order to do this more efficiently I thought I’d bring some of the logs into a simple MySQL database. As a result I had to write this SQL table definition to create the table to import in, and seeing as it just took me about an hour to do I thought I’d share it with you and help you save that extra hour that you could be using to work out why that expandable isn’t working on on your iPad application using the DFP Mobile SDK…. good luck.

UPDATE: 2013-11-20 – Had to make some of the ID fields BIGINT instead of INT as some integers such as CreativeId was exceeding 2147483647.

CREATE TABLE `LOG_IMPS` (
`Time` datetime NOT NULL,
`UserId` varchar(24) DEFAULT NULL,
`IP` varchar(8) NOT NULL DEFAULT ’0.0.0.0′,
`AdvertiserId` bigint(11) NOT NULL,
`OrderId` bigint(11) NOT NULL,
`LineItemId` bigint(11) NOT NULL,
`CreativeId` bigint(11) NOT NULL,
`CreativeVersion` int(11) NOT NULL,
`CreativeSize` varchar(64) DEFAULT NULL,
`AdUnitId` bigint(11) NOT NULL,
`CustomTargeting` varchar(2046) NOT NULL,
`Domain` varchar(64) DEFAULT NULL,
`CountryId` int(11) DEFAULT NULL,
`Country` varchar(32) DEFAULT NULL,
`RegionId` int(11) NOT NULL DEFAULT ’0′,
`Region` varchar(32) DEFAULT NULL,
`MetroId` int(11) NOT NULL DEFAULT ’0′,
`Metro` varchar(32) DEFAULT NULL,
`CityId` int(11) NOT NULL DEFAULT ’0′,
`City` varchar(32) DEFAULT NULL,
`PostalCodeId` int(11) NOT NULL DEFAULT ’0′,
`PostalCode` varchar(16) DEFAULT NULL,
`BrowserId` int(11) NOT NULL DEFAULT ’0′,
`Browser` varchar(32) DEFAULT NULL,
`OSId` int(11) NOT NULL DEFAULT ’0′,
`OS` varchar(32) DEFAULT NULL,
`BandWidth` varchar(32) NOT NULL,
`BandwidthId` int(11) NOT NULL DEFAULT ’0′,
`TimeUsec` int(11) NOT NULL,
`AudienceSegmentIds` varchar(1024) DEFAULT NULL,
`RequestedAdUnitSizes` varchar(32) NOT NULL,
`BandwidthGroupId` int(1) NOT NULL,
`MobileDevice` varchar(32) DEFAULT NULL,
`OSVersion` varchar(32) DEFAULT NULL,
`MobileCapability` varchar(32) DEFAULT NULL,
`MobileCarrier` varchar(32) DEFAULT NULL,
`Product` varchar(32) NOT NULL,
`IsCompanion` varchar(16) NOT NULL,
KEY `AdvertiserId` (`AdvertiserId`),
KEY `OrderId` (`OrderId`),
KEY `LineItemId` (`LineItemId`),
KEY `CreativeId` (`CreativeId`),
KEY `CreativeSize` (`CreativeSize`),
KEY `Country` (`Country`),
KEY `MobileDevice` (`MobileDevice`),
KEY `MobileCapability` (`MobileCapability`),
KEY `MobileCarrier` (`MobileCarrier`),
KEY `BrowserId` (`BrowserId`),
KEY `Browser` (`Browser`),
KEY `OSId` (`OSId`),
KEY `OS` (`OS`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

Now, some of the above VARCHAR and INT sizes will probably need tweaking but with the above I managed to bring in a subset of one of my impression logs. As I play with the other Event logs and click logs I’ll post up table definitions for these as well.

November 13th, 2012 | Tags:

Admonster’s Maria Tucker has done a great job of bringing together a raft of insightful knowledge and links regarding the current state of play in Mobile Malvertising. It’s a great starting point for any conversation around this area.

Read the article here.

October 3rd, 2012 | Tags:

Not much activity from me of late, my apologies. But the wait may be wroth it.

I ran across this new product from Google today: https://www.google.com/tagmanager

Maybe not as feature rich as the rest BUT free and relatively easy to setup The rules engine seems limited at the moment but looks likely to be extended in the future. Scripts delivered from www.googletagmanager.com in case you want to make sure no-one’s adding these to your 3rd party adtag.

July 24th, 2012 | Tags:

No posts in quite a while, things have been rather busy. In the meantime check out this latest video from our friends over at Improve Digital.

Finding suitably in-depth, accessible and free training materials for iOS/iPhone/iPad etc is something I’ve been looking to find for a long time now. Understanding the development lifecycle as well as the opportunities this platform offers publishers (specifically relating to the advertising opportunities they provide us) is an area I have struggled with in the past year. That’s why I’m very happy to see a number of Universities are making complete course lectures/tutorials available online (mainly through iTunes University) and in some instance all handouts and associated code/projects. This is to be highly commended and is a position I would like to see more colleges and institutions adopt.

Arguing for training resource is something that we really shouldn’t have to spend too much time doing. This is a fast moving industry with new technologies, concepts and business practices seemingly emerging every other week. If we don’t keep ourselves abreast of these changes and their potential impact on our business operations then who else will? But, the realities are that new technology will get released after training budgets have been agreed and then we find ourselves spending valuable time on the cost-benefits rather than concentrating on educating and equipping our staff/teams and bringing insights & knowledge to the wider business.

Now, you’re probably asking yourselves what learning iPhone programming has to do with Advertising Operations, after all it’s not like you’re looking for seasoned C++ developers when recruiting a new Trafficker or requiring your Campaign Managers to have a Degree in statistical methodologies. But my argument has always been that AdOps lies at that intersection of Technology/Development, Advertising Theory, Statistics/Analytics and Business Development. When new technology rears and the business want to know how to leverage advertising on it then we’re the ‘go-to’ people. When there is no-one else in the company who can give a view on some latest gadget or have an opinion on how such technologies can be brought under the wider advertising strategies… you need to speak to AdOps. Keeping abreast of technological change is second nature for AdOps people, if your business doesn’t have someone with AdOps experience who is consulting with your Business Development group then you’re missing a trick.

This isn’t about getting your staff to become iOS developers, it’s about educating them to understand how these devices work in order that they are then able to advise how best to implement advertising technologies.
The more your staff are empowered by this knowledge and understand how such systems work, then the easier it will be when managing the impact of their uptake by your customers. Ultimately this should feed into the overall approach your company takes in reacting rapidly to such huge changes in the technology landcape.

All of the following courses contain between 10 and 13 videos each of which last approx 1-2hrs and will require access to a Mac running Xcode.

iPhone Application Development [Stanford]
http://itunes.apple.com/itunes-u/iphone-application-development/id384233225#ls=1

Developing Apps for iOS [Stanford]
http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewPodcast?id=395631522

iPhone Application programming [RWTH Aachen]
http://itunes.apple.com/itunes-u/iphone-application-programming/id390395304

Advanced iPhone Development [Madison Area Technical College]
http://itunes.apple.com/WebObjects/MZStore.woa/wa/viewPodcast?id=407243032

November 22nd, 2011 | Tags: , , ,

I ran across this presentation slide-deck recently. It provides a thorough insight into the their underlying platforms, technology & infrastructure implementation (including some screen shots of their internal cloud management platform, based on Amazon’s AWS).

But what has this to do with Advertising Operations? Well, I feel the similarities occur when thinking about building scalable & highly redundant/available systems (as they say themselves, they value availability over consistency). There’s quite a deep dive into their decision about using Cassandra which I found insightful and educational and well worth trying to get to grips with as it’s becoming increasingly clear that the whole NoSQL field (certainly in so far as the amount of data we are able to generate within online-advertising systems) has many benefits over traditional DB systems.

November 17th, 2011 | Tags: , ,

Whilst discussing the ‘Data’ issue with my boss recently we referred to Ben Kneen’s excellent blog AdOpsInsider, specifically to a series of articles he wrote back in July concerning Data Management

Data Management Part I: What Are Data Management Platforms?
Data Management Part II: Centralize and Synchronize Your User Data
Data Management Part III: Syncing Online Data to a Data Management Platform
Data Management Part IV: Syncing Offline Data To Your DMP

I’d thoroughly recommend reading these as they are a great introduction to the whole data/DMP field from a specifically AdOps perspective. Irrespective of whether you currently have any plans or strategies around leveraging the data your various adserving, analytics & behavioural platforms provide, these articles will give a solid view of what and how these technologies can start to work together.

November 15th, 2011 | Tags:

(Via MalwareDomainBlocklist.)

Bash commands to detect script injections and malware:
This was posted a while ago on stopbadware and it’s too good not to repost…

find . -name “*.js” | xargs grep -l “eval(unescape”
find . -name “*.php” | xargs grep -l “eval(base64_decode”

The first one will find any javascript file that contains the string “eval(unescape” which is the most common way of injecting malicious code. The second is a similar method for PHP files. (Source: https://badwarebusters.org/stories/show/20712)