Media companies, don’t EVER trash your content.

At one point in my career, I contributed an article to the publication I was working for at the time. Nothing fancy, just an interview with an international artist. The publication killed the story, and I never saw it again. I’d publish the story online myself, but the article still legally belongs to the publication. After all, I would not have scored the exclusive interview I needed for it without their help. I don’t think the publication should publish it or post it online to protect my feelings, though that might become a concern with a bigger-egoed writers. I don’t care either way. This article, however, took me weeks of research and use of the publication’s resources to write, and because it didn’t fit the format they were looking for—and frankly because another more seasoned writer had a better article on a similar story—the publication scrapped it.

When a publication scraps any content, it’s like they’re dumping money into the trashcan. Even if an article doesn’t suit an issue or format, it doesn’t mean that piece of content won’t hold any value to someone at some point in time. Online access to this content helps the publication actualize their content’s potential value. Tagging and linking to each piece of content helps interested users get to this content easier.

I’m an archivist at heart. I want to preserve my work and the works of my colleagues for centuries to come, and I want to facilitate the access to these works so the most information reaches as big of an engaged audience as it can over time. But content by itself does not have any agency. As content producers and providers, we must not only post our content online but construct an appropriate mechanism that will automate how this information is shared and linked to other relevant information online, be it our own or someone else’s.

Easier said than done, for sure, but a library of resources exists online to make projects like this mechanism happen (think API tools, XML and script developer forums or CMS/middleware development communities). Once this mechanism is built—and this is key—the publication needs to maintain the mechanism’s flexible, open-source powered structure so that the service can be continually streamlined to meet users’ growing needs.

Nothing online is set in stone. In a year, web developers might have completely moved on to punchier and sleeker solutions such as CSS3 and HTML5 for some of their web development needs until some other collective improvement comes along to enhance web development even further. The root question that drives these changes will remain the same, however: How do we share information in the most efficient way possible, for developers and users?

If media companies, publications and journalists start thinking of their content as nodes in need of relevant connections to other content nodes online—or even better, think of accurate descriptive keywords and other data atoms composing their information—then this question will be much easier for them to answer.

Right now Parallactic Drift stands more as a proof of concept of a scalable AJAX framework that can fit into any CMS that can automate its compiling XSLT scripts (through either Cron or Cron-like functions) and a better search function.

The framework is entirely client-side, using jQuery to hide and present search query-matching stories in individual modal windows that float above a fixed map with location points. As a result, it’s a pretty big page with a slow search function and other limitation (i.e. the page does not contain my favored fade others function for emphasis because of the sheer number of news items). The news items come from a few weeks of gathering all of the RSS feeds from the BBC.co.uk, Economist.com, English.Aljazeera.com and CNN.com. So individually, the news items number in the thousands.

Please forgive the bugs and the lack of browser compatibility of this version. That said I’m still working through some of the kinks (such as not getting all of the results to show up during a search), and I welcome any constructive criticism through the comments section here, my twitter or through my portfolio site‘s contact form.

I’d like to give special thanks to Jim Zullo, Vin Crosbie, Matt Doxtad and the host of real programmers online at Stack Overflow for helping me make this beta launch possible.

Like Derek, I want to help journalists solve redundancies within their information gathering and distribution models. All of the hard data that many of my hard-working colleagues gather (i.e. names, ages, dates, etc.) should be stored at the most granular level within a relational or hierarchical database should be easily reused and accessible to everyone, not just journalists. Practical sorting of individual articles, i.e. by relevance, relies on an article’s meta-data, the most accurate of which is derived from the article’s most minor elements.

While I can understand why the publication does not give its own developers access to their information-rich database, I think this resistance to share access to information is symptomatic of the deep separations that still plague the news industry. In particular, a wall has been erected that should not be there: the wall between the IT and the editorial departments. IT people are almost always overloaded with work already. One IT employee often has to maintain intra-office networks, develop web sites, maintain different CMS applications, which are most likely tweaked from a mainstream solution or made entirely from scratch for the publication and parent company, if one exists.

This tendency to bundle positions and overload technology workers has extended to online journalists, who now increasingly take on multiple positions such as web developers, designers, videographers, et. al. along with their editorial responsibilities. As a result, the average new media department’s workflow alienates journalists from their editorial duties, as well as the publication editorial department from the publication’s online products. In order to establish a viable model for web content creation, publications have to restructure their workflow in a fundamental way.

To maintain a good quality site, publications cannot be afraid to experiment in a range of different areas. Journalistic publications must not only staff these positions with reasonable expectations from each employee, but they must give these employees enough latitude to experiment in new projects and find out what old and new techniques thrive online, thus also allowing them to ditch whatever doesn’t work before it becomes a massive drain on time and money.

This level of autonomy is akin to the one necessitated by reporters and editors. Perhaps more than most other professions, journalists rely on a careful system of trial and error, one in which problems must get corrected as soon as they come up so that the next day’s/week’s/month’s/quarter’s product is better. This facet of journalism makes it a natural fit with how development and information sharing works in the World Wide Web.

Large media companies such as NYT and LATimes can afford to invest a lot of research and development without having to commit the entire publication’s to a new media, at least not as much as small- or mid-sized publications. But a large company cannot change at a fast enough rate to address the changing needs of its audience. And I don’t think such a company should be expected to remake itself every few months or years. What a publication needs is a new media department structure that is flexible enough to allow online journalists, developers and designers to have the level of autonomy print journalists have. A publication must allow all of its individual employees to take risks and fail so they can sometimes succeed.

An attempt to present all the countries and cities that appear in the scraped RSS XML makes for a drop-down or scroll-down menu that is way too long to be practical. Also, instead of using a menu for sorting stories by format (text, video, audio, et. al.) I’d rather use the simple icon key explained in that same post. Thus, menus for my project are a no go for now.

Another minor issue I’d like to work out is how to present overlapping file types. For example, I want to be able to present the link to a story that has text with an embedded video as having video and text. So far I’m thinking of using another so-called “code scraper” such as HtmlCleaner, which turns HTML code into plain text.

The advantage of this kind of service is HUGE since HTML is often neither consistently coded nor well-formed enough to look for meta-data in a structural way (unless you’re a Google crawler). Reducing code to text gets rid of that factor and makes literal expressions much easier to find. For that video and text example, I can just ask to look for beginning of the literal embed code for a YouTube video (i.e. ‘embed src=\&qt\http://www.youtube.com/v’).

My main problem is that I can’t use a parser that needs to store these text files. Remember, I’m trying to do this with no relational database, so storing these plain-text applications is not a viable option for me. That said, if someone knows of a way to read HTML as plain text with pure JavaScript or XSLT, please leave a comment — I’m all ears.

Thus, I will employ JS to call the appropriate search and query functions in the XSLT script. For instance, when the term “Israel “is called, the appropriate items containing this term will appear, from which I can draw metadata to style a plot point on a world map and arrange descriptive information (i.e. headline and lede) into separate modal boxes.

For those of you who are not sure what AJAX means—I know I didn’t at first—this intro video on AJAX from the good people at SitePoint can really help explain what the buzz term means, as well as each and every function that makes it work. Primarily operated by Canadian developer and technical author Kevin Yank, SitePoint’s provides very concise, plain-spoken tutorial and overview videos for newbies like me. I only wish I had the ability to communicate my own project in such lucid terms.

News Facet’s project also differs from mine since the developers of the site opted to display the most recent stories, only going back as far as 30 hours. With Parallactic Drift, I want to bring focus to even older stories, since access those stories can only grow over time and a well-constructed and well-presented XML framework that groups these stories by different relevance factors will promote their accessibility to online users.

Still, News Facet has a sharp and organized design that takes advantage of many different free APIs available online. I encourage anybody who wants a better idea of what I want this project to look like to check out NewsFacet.com.

Although I include the publication date as part of each item’s presentation, it will not bear any impact the site’s organizing principle. Each item’s story location(s) will also show up as a dot on a world map. Hovering over either a dot or its corresponding news item’s box brings focus to both the news item and the dot (i.e. by fading out all other story items and dots).

Lowering the dots’ opacity to about 5%, which is achieved through with a simple RGB command such as “opacity:.05,” also allows for another way to topographically display the semantic relevance among different news items. The more stories in a news archive that have a specific location in common—such as the Maule Region in Chile and Port-au-Prince, Haiti stories for the search term “earthquake”—the more opaque a dot appears in those locales. This practice can also make the site more interesting for general browsing by topic and perhaps someday as an archived news stream, similar to RSS feeds but presented in a much more contextual and interesting way.

So far, both the XSL loop and the CSS for this design are coming along smoothly. Now I need to work on some Javascript to make the query loop (or the over-arching conditional that will return news items from the XML database) connect to a jQuery drop-drop menu and case insensitive search bar. As with XML, I’m pretty new at Javascript, but I’m never afraid to learn more coding. I think I can pull this project out within the coming month and have a nice framework that I will be proud of having created.

Even as an XML newbie, I’ve been able to streamline the workflow I highlighted two posts ago. I’ve opted for a massive, well-constructed XML file instead of a folder structure. The XSLT stylesheet I’m coding now will draw the necessary information from the XML file and place each element in a uniquely labeled div, which I will later style with CSS.

To eliminate any duplicate news items, I’ve turned to the admirable open-source work found at EXSLT.org. This community has created extensions that help simplify overly complicated XSL commands, in this case reducing a complicated template command to the line set:distinct().

I still intend to make a database of locations and file types, but I think coding a discrete topic database by hand. The more I think about it, the more I want the topic navigation to be user-defined (i.e. with a search bar) rather than solely defined by me, the developer (i.e. with a drop-down menu with a discrete set of terms).

For anyone who is interested, my XSL loop (with comments) currently looks like this:

<xsl:for-each select=”set:distinct(channel/item)”>

<!–This is the query loop. Query terms can be keywords, locations or file-types.–>

<xsl:if test=”contains(title,’Israel’) or contains(description,’Israel’)”>
<div>
<div><a><xsl:attribute name=”href”><xsl:value-of select=”$link” /></xsl:attribute><xsl:value-of select=”title” /></a>
</div>

<div><xsl:value-of select=”substring(pubDate,1,16)” /></div>

<div><xsl:value-of select=”description” /></div>

<!–Replace each “text,” “video” and “audio” below with appropriately attributed img src tag–>
<div>
<xsl:choose>
<xsl:when test=”contains(type,’video’)”>
Video
</xsl:when>

<xsl:when test=”contains(type,’audio’)”>
Audio
</xsl:when>

<xsl:when test=”contains(type,’audio’) and contains(type, ‘video’)”>
Audio Video
</xsl:when>

<xsl:when test=”contains($channeldescription, ‘column’) or contains($channeldescription, ‘views’) or contains($channeldescription, ‘analysis’)”>
Opinions
</xsl:when>
<xsl:otherwise>
Text
</xsl:otherwise>
</xsl:choose>
</div>

<!–This div will reveal the source’s location for each item.–>
<xsl:choose>
<xsl:when test=”contains($channeldescription, ‘BBC’) or contains($srcloc, ‘bbc’)”>
<div></div>
</xsl:when>
<xsl:when test=”contains($channeldescription, ‘CNN’) or contains($srcloc, ‘cnn’)”>
<div></div>
</xsl:when>
<xsl:when test=”contains($channeldescription, ‘Economist’) or contains($srcloc, ‘economist’)”>
<div></div>
</xsl:when>
<xsl:when test=”contains($channeldescription, ‘Jazeera’) or contains($srcloc, ‘jazeera’)”>
<div></div>
</xsl:when>
</xsl:choose>
<!– This loop, which I will more fully populate later, determines the story’s location and return the appropriate “loc” div to style–>
<xsl:choose>
<xsl:when test=”contains(title, ‘Israel’) or contains(description, ‘Israel’)”>
<div></div>
</xsl:when>
</xsl:choose>
</div>
</xsl:if>
</xsl:for-each>

I’m considering OpenCalais as a sorting tool for Parallactic Drift. Believe me, hard coding keywords and conditionals to match specific topics is difficult, but it allows me to create a script that sorts news items down to specific events. While I would lose this level of specificity with OpenCalais, the API seems to have evolved from its initial functions to include more advanced vector calculations for reading word placement within text. The API even diagrams sentences’ grammatical structure to figure out what a sentence says! Just type a word into their demo viewer.

I’m thoroughly impressed with the ingenious and collaborative energy behind this project, and I feel like utilizing it in the project just to contribute to their ever-expanding database. Even if I don’t end up using any of OpenCalais’s tools, I certainly recommend at least their CMS tools to online publications and most other site developers, since it can only help make everyone’s meta-information more descriptive and content more findable.

My professors assure me that this mechanism is sort of innate to the coding languages I will be using, and that I can create a robust archive with simple conditional (if this, then that) commands. My limitations lie in my programming capabilities. I know I can’t create an automated system simply because I can’t pick up a complex programming language like Python and learn it within a week or even a month. This same principle goes for making a heuristic, or self-teaching, semantic aggregator that makes associations between words such as “West Bank” and “Palestinians.” This kind of programming is for the trained professionals and enthusiasts, not for code newbies like me.

What I’m doing instead will take a lot of work, however. Since I plan on working with a static number of news items, I plan on hard coding the key terms present in these stories into the scripts I plan will use to parse news items and sort them into the different folders. While I start cutting my teeth with the Java, XML and XSLT code for one of my other classes, I need to start gathering this information bit by bit now so I can hard code into the script later. This method may seem crude and needlessly laborious for the average programmer, but it reflects both my expertise and the purpose of my project (of taking a static set of archived information and making it more interactive). Besides, real programmers can always improve upon this model if it were ever implemented for a single or multiple publications, and I want to encourage this behavior.