Inside Analysis

Open Government Data Act – The Challenge

By Russell Ruggiero, Ranjeeth Kumar Thunga, and Matthew Harang

The Open Government Data Act was signed into law on January 14, 2019. The primary focus of this new initiative is for Federal Agencies to publish all of their information as open data (using standardized, non-proprietary formats). The main objective of this short piece is to help explain how the Open Government Data Act addresses important topics that include using standardized non-proprietary formats, transparency, and how open-standards play into the overall equation.

The Challenge – What will it take to realistically implement and adopt this government-wide initiative? There will be no magic bullet or quick and easy fix. We should look at the standards organizations, which include the World Wide Web Consortium (W3C), Organization for the Advancement of Structured Information Standards (OASIS), Internet Engineering Task Force (IETF), Association for Information and Image Management (AIIM), and International Organization for Standardization (ISO), for direction regarding current and future industry trends. Through their various comprehensive efforts, these well-respected organizations have helped to promote “Open” or “Agnostic” standards that in effect allow for more harmonious and seamless information exchange. From an implementation perspective, one relevant question that must be dealt with revolves around how widely used technologies, such as C# and NetBeans, along with open-standards that include JavaScript Object Notation (JSON) and Extensible Markup Language (XML), will be leveraged moving forward.

Why is Using Standardized, Non-Proprietary Formats Important?
The value of adopting standardized, non-proprietary data formats is hard to overestimate. However, it takes a broader vision to be able to grasp the true value of why to go through the process of incorporating such formats, and the benefits such formats can provide. Below are seven key reasons to consider incorporating, as well as contributing, to such standards.

1. Precise Communication – The cornerstone of communication necessitates that two or more humans, and/or machines, are actually speaking the same language. With standardized, non-proprietary formats, we can ensure that all parties are on the same page and all technologies are truly interoperating, preventing miscommunication and misunderstanding that arise from differing representations of data.
2. Efficient Work – By using standard formats, we save time and energy by focusing our attention on the data itself, rather than how to best represent or capture it. No need to reinvent the wheel – we can quickly proceed with subsequent steps, taking advantage of the work that’s already been done in fleshing out the appropriate ontological frameworks for us to use.
3. Preexisting Ecosystem – Tools, methodologies, and expertise already exist around preexisting data standards. By leveraging non-proprietary standards, we have the opportunity to also leverage resources that have been developed around such standards.
4. Cooperative problem-solving – To think and operate beyond the myopic lens of just ourselves invites us to perceive problems, and their corresponding solutions, not just in terms of what we can do, but in terms of how we can work together as a community to address challenges bigger than those we alone could tackle.
5. More Data, More Validation – Standard frameworks naturally lead to more accessible and available data. With more data available, we are opening up more of our processes to validation and verification of their efficacy. This ensures that these processes, as well as our institutions, are operating at their full potential.
6. Self-Reflection – Standard frameworks help us reflect on our own data collection. We are taking advantage of a feedback loop that informs us of what data points we might also need to include and consider in our own work, through the collective wisdom invested in the creation and development of the standards we are using.
7. Expand our Influence – When we use as well as contribute to the ongoing development of standard frameworks, we can exert greater leverage in the world at large. Our value as an individual or organization, which might have been relegated to a small sphere of influence, could instead translate and be magnified into one of much greater scale.

There’s no doubt using data standards might initially appear to challenge our conventional notions of proprietorship and control. The question however we must ultimately ask ourselves is what actually is ours to define and what about what we are capturing actually transcends us? By being open to a bigger picture, we might actually find such formats not just serve our needs better but allow us to paint on a much larger canvas than would have otherwise been possible, serving and connecting with a much greater slice of humanity in the process.

Why is Transparency Important?
The value of transparency is central to the effectiveness of a democratic republic. Citizens will benefit from the Open Government Data act, simply for the fact that is further breaks down information barriers between federal agencies and the public which it serves. In recent years, there have been many questions broached regarding the inner workings of the government, in regards to government spending, as well as specifics of involvement with corporations and other entities. With the current law in place, many of those questions will be answered with more ease.
The prevalence of the internet has caused an information overload in society, for better and for worse. Every piece of information one can imagine is available at one’s fingertips, at any time. The problem has become that there is a lot of misinformation mixed in, and it often impossible to distinguish truth from lies online. For this reason, real verifiable sources have become invaluable. In many ways, it has become more difficult to hide the truth due to the internet and social media, however, it can be just as easy to cover the truth or mask it through the same means. For this reason, transparency has become more important than ever. The public relies on real, accurate information, distinguished from “fake news” to make informed decisions about politics, policy and social issues. The Open Data act ensures that accurate information is coming directly from the source.
The fact is that government corruption is a global problem, and the United States is not immune. Politicians and policymakers alike are constantly involved in back door deals, illegal transactions, and even legal deals that blur the lines of ethics. Before the internet, it was much easier to hide such actions from the public, because even when discovered, the news would only spread locally. With the ever-increasing usage of social media outlets, news of such wrongdoings spreads like wildfires. The Open Government Data act ensures that these government entities will continue to be held accountable for their actions. It ensures that they will act with discretion, knowing that ever action will be under the scrutiny of watching eyes. The law enables citizens to have the assurance that everything done behind closed doors can come to light, if need be. No actions can be hidden from public view, nor swept under the rug “to protect the feebleminded.” This is an important step to ensure accountability at all levels, and due provide a method of deterrence of the abuse of power.

Why are Open-Standards Important?
The implementation of the Open Government Data Act will require a forward thinking mindset which uses building block technologies, along with an innovative group of public and private sector solutions, in such a way as to foster a truly agnostic Federal ecosystem. Hence, if we focus on the principal goal of “using standardized, non-proprietary formats” at face value then open-standards will play a crucial role in moving the Open Government Data Act from a vision to a reality. Accordingly, we should look to organizations such as W3C, OASIS, IETF, AIIM, and ISO for guidance regarding the direction of both current and emerging open-standards. It is important to note that these organizations encourage a “world-wide” perspective regarding deployment, and one that is not encumbered nor predicated by physical boarders. In effect, this pragmatic and pliable approach allows for accepted open-standards to be leveraged across the globe. It means moving away from proprietary technologies that are most often “vendor specific” to a true agnostic platform that will allow for a modernistic and versatile “plug-and-play” implementation methodology.

As a result, there is a real need to agree on open-standards that in fact foster seamless bidirectional and unidirectional information flow between all interested parties. We must then try to envision Federal Agencies actually agreeing on reducing the number of prime technologies used to help streamline the ecosystem system. Using standardized, non-proprietary formats could also shift important valuable resources away from supporting a myriad of redundant and, or antiquated systems, to the implementation of current and future “agnostic” technologies to help push the communications envelope. Consequently, this positive development would in effect enable these entities to create an infrastructure that could better deal with the day-to-day operations, along with improving the response to manmade (e.g., terrorist attacks, oil spills, etc.) or natural (e.g., hurricanes, fires, earthquakes, etc.) catastrophic events. Because of the sheer magnitude and number of recent catastrophic events, using standardized, non-proprietary formats should be a priority and not an afterthought regarding the implementation of the Open Government Data Act. On that note, how will emerging relevant open-standards such as Emergency Data Exchange Language (EDXL) blend into to mix?

If properly implemented, the Open Government Data Act will usher in a new era regarding how data is shared not only among Federal Agencies, but also to the public. Information Technology has currently evolved to the point that a shift away from proprietary to standardized formats is indeed possible. Bottom line, the result will be greater transparency, along with improved overall efficiency. Another positive benefit that we can expect, and not to be overlooked from this new initiative, will be the many important innovations hatched by the private sector. All in all, implementation the Open Government Data Act could be looked at as a daunting challenge, but it could also be viewed as a great opportunity to improve the Federal ecosystem.

Key to links:
World Wide Web consortium (W3C) www.w3.org
Organization for the Advancement of Structured Information Standards (OASIS) www.oasis-open.org
Internet Engineering Task Force (IETF) www.ietf.org
Association for Information and Image Management (AIIM) www.aiim.org
International Organization for Standardization (ISO) www.iso.org
JavaScript Object Notation (JSON) http://json.org/
Extensible Markup Language (XML) http://www.w3.org/XML/
Emergency Data Exchange Language (EDXL) https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=emergency

15 Responses to "Open Government Data Act – The Challenge"

  • Rex Brooks
    March 18, 2019 - 11:09 am Reply

    It should be noted that that the Emergency Data Exchange Language (EDXL) is the composite collection of topic-specific standards and specifications built by an open process in OASIS (see Key Links above) by the Emergency Management Technical Committee (EMTC) over the last 16 years.
    EDXL Suite of Standards now includes:

    • The Common Alerting Protocol v1.2 specification (EDXL-CAP), with various dedicated profiles
    • The Distribution Element specification v2.0 (EDXL-DE)
    • The Hospital AVailability Exchange specification v1.0 (EDXL-HAVE)
    • The Tracking of Emergency Patients specification v1.1 (EDXL-TEP)
    • The Resource Messaging specification v1.0 (EDXL-RM)
    • The Situation Reporting specification v1.0 (EDXL-SitRep)
    • The Tracking of Emergency Clients v1.0 (EDXL-TEC)

    All of the documentation for these standards can be found at http://docs.oasis-open.org/emergency/

    Work is underway on an article exploring EDXL in OGDA context.

    • Russell Ruggiero
      Russell Ruggiero
      March 18, 2019 - 4:04 pm Reply

      As always, you are excellent at connecting the dots. Trying to create a seamless and agnostic Federal IT ecosystem would allow for improved bidirectional communication flow between agencies in dealing with manmade (e.g., oil spills, terrorist attacks, etc.) and natural (e.g., earthquakes, hurricanes, etc.) catastrophic events. Accordingly, taking the open-standards route will help get there from a technology interaction
      and, or coexistence point of view. In relation to the public sector, it will be open, machine readable solutions like EDXL that will help reach the goals of important initiatives like the Government Performance and Results Act (GPRA). Case in point: EDXL could enable entities comply with Section 10 of the GPRA Modernization Act (GPRAMA), particularly now that Office of Management and Budget (OMB) M-13-13 has reiterated the direction previously set forth in Circular A-119 to use voluntary consensus standards whenever possible.

      Enter Open Government Data Act (Signed into Law on 1/14/19), which is for Federal Agencies to publish all of their information as open data (using standardized, non-proprietary formats). It is here where important open-standards like EDXL will be leveraged in the critical area of Emergency Management.

    • Gannon (J) Dick
      March 19, 2019 - 4:52 pm Reply

      Rex & Russ,

      I’ve never seen the actual sales lead records used to conduct a targeted advertising campaign over the web … but I can say that records for outbound telemarketing were batched for the telephone switch. The switches relied very heavily on the post hoc ergo propter hoc fallacy as does targeted advertising.

      No time at all is spent on resource curation. Nonetheless, this is exactly what EDXL-RM must do before the D is E. For example; both FEMA and the CDC both use the same Regional Office structure (https://www.atsdr.cdc.gov/dro/index.html). Reusable maps are easy enough to make. NOAA Fisheries and The Office of Inspector General have different Regional views, and the IRS yet another, but no matter, once one structure is defined another is a simple copy.

      • Rex Brooks
        March 21, 2019 - 12:53 pm Reply

        Gannon,

        EDXL-RM is perhaps the most needed and least used of the EDXL suite. When asked for guidance related to datamodels for use with EDXL-RM, I refer people to the Resource Typing Library Tool from DHS/FEMA, with specific mention of the Incident Resource Inventory System (IRIS), a distributed software tool provided by the Federal Emergency Management Agency (FEMA). IRIS is free and available for use by all agencies, jurisdictions, and communities to serve as a consistent tool to inventory resources into their own database and to search/identify their specific resources for incident operations and mutual aid purposes.

  • Gannon (J) Dick
    March 18, 2019 - 4:10 pm Reply

    I want to talk about JSON and XML but I found this fundamental too crucial to pass up … “The question however we must ultimately ask ourselves is what actually is ours to define and what about what we are capturing actually transcends us?”

    Or predates us. The first published use of Postal Codes in North America is likely Ben Franklin’s famous “Join, or Die.” Cartoon from 1754 (http://www.loc.gov/pictures/item/2002695523/). The four future state codes (NJ,NY,NC,SC) are there for all to see. “Open Government Data” one might say.

    Might this be the “what about” that is confusing Artificial Intelligence ?
    (https://s26162.pcdn.co/wp-content/uploads/2018/02/b510423d3f86b92884dc21fe1eb29cee-758×1024.jpg)

    • Russell Ruggiero
      Russell Ruggiero
      March 20, 2019 - 7:50 am Reply

      Gannon, thank you for the comments. I would like to address JSON/XML 101 and AI.

      JSON – JavaScript Object Notation is an open-standard format that uses human-readable text to transmit data objects consisting of attribute value pairs. It is built on two structures:

      1) A collection of name/value pairs. In various languages, this is realized as an object, record, struct, dictionary, hash table, keyed list, or associative array.
      2) An ordered list of values. In most languages, this is realized as an array, vector, list, or sequence.

      XML – Extensible Markup Language is a non-proprietary subset of SGML. It is focused on data structure and uses tags to specify the content of the data elements in a document, while XML schemas are used to define and document XML applications. Web services are components that reside on the Internet and have been designed to be published, discovered and invoked dynamically across various platforms and unlike networks.

      There are many important factors (e.g., deploying, communicating, storing, etc.) that should be taken into account. Regarding data interchange, open-standards that include JSON and XML, all have a number of positive attributes along with various limitations. We live in a world were both are used extensively, which is why we need to examine how they both can both co-exist.

      AI is getting a lot of press, as it should. Robotics, recurring neural networks, and artificial intelligence (AI) are all related. We need to take a step back and look at the “pegboard” and what “plugs” into it. An AI solution, say facial recognition system at an airport (Internal/TSA) should integrate seamlessly into “pegboard” if built to standard and, or standards (theoretically). Hence, creating a pliable, secure, and scalable Federal ecosystem that supports this “plug and play” methodology.

    • Russell Ruggiero
      Russell Ruggiero
      March 23, 2019 - 8:48 am Reply

      Gannon, want to address “what about” from a future perspective.

      We study Thales, Socrates, Plato, and Aristotle to gain a better understanding of what it means to be a Citizen. Let us stop at Aristotle and view his star pupil, Alexander the Great, who died in 323 BCE and ushered a new era in human history, the Hellenistic Age. This era created what is known at the “City State” which most often had their own political system and currency. However, these independent entities did trade with one another and sometimes even worked together in helping to thwart common foes. As we look at the City State from a holistic view, it was a functional entity that could deal with day-to-day activities, while trying to handle manmade and natural catastrophic events.

      Enter the Demokratia of the United States of America in the 21st Century. The definition of “The Citizen” would be recognizable to Thales, Socrates, Plato, and Aristotle which is a testament to this form of government. The foundation of which is based on defined laws which “The Citizen” must follow to foster harmony and to help promote organization. What differs mostly in 5th Century Athens and the Washington D.C. of today is technology. It is not only the differentiator, but also the enabler. As a result, the game plan is for technology to help improve data transparency, but also as with the Hellenistic City States to deal with day-to-day activities, while coping with manmade and natural catastrophic events.

      Let us for a moment look at Federal Agencies as City States in regards to Information Technology and their relation to the ecosystem. They are fully “Functional” and can interact when needed with other agencies. Case in point: In the event of a terrorist attack (sarin gas) the DHS/FEMA will work with the DOJ in order to find out who caused the attack.

      Fast forward 100 years, as one would expect, the Federal ecosystem will look much different than the one we know today. Yes robotics, neural networks, and artificial intelligence (AI) will play an important role, but it will be the foundation that will allow these types of solutions to integrate across the entire ecosystem. A pliable, secure, and scalable foundation that will not only improve efficiency, but also data transparency. How will this happened? Through careful planning and leveraging open-standards by the public as well as the private sectors. This type of effort will take decades, but if properly implemented and adopted will create a 22st Century Federal ecosystem that will have the ability improve the life for “The Citizen” and the entire nation.

      • Gannon (J) Dick
        March 23, 2019 - 2:50 pm Reply

        At least Alexander the Great had lousy maps. GPS makes conquering the known world political map much harder. True, India is a long walk from Macedonia, but the “knowledge” of places beyond India deforms the phrase “known world” leaving the verb “to conquer” intact.

        If you read the Mayflower Compact you’ll see the Pilgrims thought Cape Cod was in Virginia. Setting aside the enormous consequences for Spring Break destinations, the fact is maritime power Britain hit a land wall called the Americas. The British measured commercial success in degrees longitude made good. Guns, Germs and Steel still worked but with diminished capacity for the guns and steel. In any case, the age of sea discovery was paused.

        Germs are making a comeback (Flu, Measles), natural disasters (Hurricanes, Cyclones, Floods) will always be with us; Humanitarian Aid delivery is a necessity not a luxury.

        Building a healthy federal ecosystem will require two things:
        1) Federal Agency counter-measures can ill afford ignorance and political bigotry about the size of the “known world”. Lousy maps worked for Alexander the Great.
        2) “Web Discovery” like combating a Measles outbreak at O’Hare Airport or sending Humanitarian Aid to Puerto Rico or Venezuela is explicitly an overland effort. It ought to be much more Lewis & Clark and much less Captain Cook in vision.

        Is that so hard it will take decades ? Maybe so, maybe not.

        • Russell Ruggiero
          Russell Ruggiero
          March 24, 2019 - 3:00 pm Reply

          Gannon,

          Cartography issues aside, a lot of the good stuff was lost during a fire in the Library of Alexandria in 48 BCE. In response to your statements One and Two, below are the responses.

          One/Response
          Federal Agencies counter-measures can ill afford ignorance and political bigotry about the size of the “known world”. No doubt, there will be a “convergence” regarding the size of the “known world”, but this consensus will be over time and fostered by a collaborative effort by all interested parties. Sorry to be so nebulous, but technology is not the problem, it really boils down to coming to an agreement (by who and for who?) on defining the lines and boundaries.

          Two/Response
          “Web Discovery” like combating a Measles outbreak at O’Hare Airport of sending Humanitarian Aid to Puerto Rico or Venezuela is explicitly an overland effort. Your measles outbreak at O’Hare airport is quite revealing, and the one I will focus on in my response. I am no CDC expert and get my information via articles, along with TV. One does not have to search far and wide to know that measles and mumps outbreaks are common across the U.S. at high schools and colleges. Let’s circle back to the airport scenario. A couple has a layover at O’Hare on their way home from Europe. They hang out at the gate for three hours and are exposed to person sitting next to them who has a case of the measles. Afterwards, they board a flight for Minneapolis/St. Paul airport and then after landing take the light rail to the Hennepin stop. They are then picked-up by a friend who proceeds to drive them to a BBQ with some ten people attending. From Point A to Point B to Point C, so on and so forth. The spread of a disease can be predicted by computer modeling, but combating an outbreak at a hub (airport, train station, bus station, etc.) seems to be a more difficult problem to solve. It all starts with Strategic Plans/Action Plans/Performance Plans. With that do we look at an open-standard like Strategy Markup Language (StratML) or Emergency Data Markup Language (EDXL), or both?

          On a final note, you are correct and I am wrong. The U.S. has many tech-hubs (e.g., Silicon Valley, Silicon Alley, etc.) that can help with the heavy lifting and the measurement of time should be in years and not decades.

          Russ

  • Charles S. Assaf
    March 25, 2019 - 1:30 am Reply

    Transparency can really only be guaranteed by the use of Open Standards. You simply cannot take the word of a profit-driven manufacturer that they’ve exposed all APIs and other external interfaces. Hidden access methods can not only provide unfair advantages for some, but also the ability to make unseen manipulations to data that can compromise its integrity. By adhering to an Open Standard, the structure of and access methods to that data are freely available to those that need to know.

    Take this query that was recently posed to me: How do we deal with widely used technologies like C# and NetBeans, along with open standards like JSON and XML.

    Firstly, C# is a programming language, and NetBeans is an IDE (Integrated development environment). This is not a comparison of apples to oranges here – it’s comparing apples to the cart they are delivered in.

    With respect to open standards, both support them. C# is a mature Microsoft programming language, and supports both XML and JSON (as long as you do it Microsoft’s way which generally requires the non-open .Net architecture). NetBeans, being an IDE, is a container that allows you to work with Open Standards, Microsoft’s proprietary standards, or other standards in the industry.

    Netbeans, and C# to some extent, are also “noughty” technologies, IMH. They are dated and losing relevance, as more advanced frameworks for Javascript and PHP are providing cross-platform capabilities, which not only support Open Source, but are often driven by Open Source projects. And, I personally switched from using NetBeans a couple of years ago to Microsoft’s Visual Studio Code as my IDE (ironically).

    • Rex Brooks
      March 25, 2019 - 9:18 am Reply

      Just BTW, NetBeans is based on Java, not C# and NetBeans is in the process of being transferred to Apache. I also switched to Visual Studio and C# when this process started more than a year ago.

      Rex

    • Matthew Harang
      March 26, 2019 - 11:31 pm Reply

      Charles, you’re right on. Transparency can only be achieved with open standards, and we have to work within the technological perimeters to achieve true transparency without loopholes. The law as it stands is a step in the right direction, but as with all legislation, it must be tweaked and updated as new technologies and standards develop and emerge. This will be an interesting topic to follow in years to come.

      • Gannon (J) Dick
        March 27, 2019 - 2:44 pm Reply

        I agree that transparency can only be achieved by open standards. I would include in open standards authoritative standards. Way back when I bought the CRC Handbook of Chemistry and Physics although a current print copy is $230, I am positive some of the tables are the same.

        The same goes for administrative entities (county names for example). But, but, but, AI will fix it right ? No, AI will fork it by recognizing nicknames and abbreviations a local would not. This is forgivable translating Arabic to English and nobody tries. But, leaving the “funny marks” out of Proper Names in Puerto Rico seems to imply a different open standard standard of rigor.

        • Russell Ruggiero
          Russell Ruggiero
          April 3, 2019 - 7:40 am Reply

          We can leverage technology to not only improve overall transparency, but also our day-to-day lives. As you pointed out, open-standards will play an important role. Why? Because they create order. Let me give you simple example. DVD’s. They follow standards and were a breakthrough technology in their day. Are all DVD’s the same? I wanted to buy a 2018 movie for my wife for her Birthday, and found the box set. I was so glad and could not wait to play them on her Birthday. DVD One: Would not play on any of my two home players, nor my son’s game consol. Fortunately, I was able to access the movie via online code. DVD Two: Came with two discs, Blu-ray and another non-Blu-ray disc which played on all three players. Bottom line, when things are not built to standards, there can be compatibility issues between human to machine or machine to machine.

          • Matthew Harang
            April 3, 2019 - 10:22 pm

            Excellent point Russ, technological standardization is becoming increasingly important in this digital world. As we move further into the realms of AI and IOT and as the 5G landscape unfolds, government standards as well as commercial standards will play a huge role in the every day conveniences that are ubiquitous as tech continues to develop. The digital landscape will change drastically over the next 5 to 10 years, and it will be interesting to see the role that such advancements play in transparency, as well.

Leave a Reply to Rex Brooks Cancel reply

Your email address will not be published. Required fields are marked *