Artificial Neural Networks and Engineered Interfaces

The question persists and indeed grows whether the computer will make it easier or harder for human beings to know who they really are, to identify their real problems, to respond more fully to beauty, to place adequate value on life, and to make their world safer than it now is.

― Norman Cousins, The Poet and the Computer, 1966

---

Grimm Brothers' delineation of the mirror answering back to its queen has breached the imagination boundaries of the fairytale in 2016. Communicating with a voice-controlled personal assistant at your home does not feel alienating anymore, nor magical.

The need to express ourselves and communicate with others is fundamental to what it means to be human. Animal communication is typically non-syntactic, with signals which refer to whole situations. On the contrary, human language is syntactic, and signals consist of discrete components that have their own meaning. Human communication is enriched by the concomitant redundancy introduced by multimodal interaction. The vast expressive power of human language would be impossible without syntax, and the transition from non-syntactic to syntactic communication was an essential step in the evolution of human language. Syntax defines evolution. Evolution of discourses along human-computer interaction is spiraling up repeating evolution of discourses along human-human interaction: graphical representation (utilitarian GUI), verbal representation (syntax-based NLP), and transcendent representation (sentient AI). In Phase I, computer interfaces have relied primarily on visual interaction. Development of user interface peripherals such as graphical displays and pointing devices have allowed programers to construct sophisticated dialogues that open up user-level access to complex computational tasks. Rich graphical displays enabled the construction of intricate and highly structured layout that could intuitively convey a vast amount of data. Phase II is currently on-going; by integrating new modalities, such as speech, into human-computer interaction, the ways how applications are designed and interacted with in the known world of visual computing are fundamentally transforming. In Phase III, evolution will eventually spiral up to form the ultimate interface, a human replica, capable of fusing all previously known human-computer/human-human interactions and potentially introducing the unknown ones.

Human-computer interactions have progressed immensely to the point where humans can effectively control computing devices, and provide input to those devices, by speaking, with the help of speech recognition techniques and, recently, with the help of deep neural networks. Trained computing devices coupled with automatic speech recognition techniques are able identify the words spoken by a human user based on the various qualities of a received audio input (NLP is definitely going to see huge improvements in 2017). Speech recognition combined with language processing techniques gives a user almost-human-like control (Google has slashed its speech recognition word error rate by more than 30% since 2012; Microsoft has achieved a word error rate of 5.9% for the first time in history, a roughly equal figure to that of human abilities) over computing device to perform tasks based on the user's spoken commands and intentions.

The increasing complexity of the tasks those devices can perform (e.g. in the beginning of 2016, Alexa had fewer than 100 skills, grew 10x by mid year, and peaked with 7,000 skills in the end of the year) has resulted in the concomitant evolution of equally complex user interface - this is necessary to enable effective human interaction with devices capable of performing computations in a fraction of the time it would take us to even start describing these tasks. The path to the ultimate interface is getting paved by deep learning, while one of the keys to the advancement in speech recognition is in the implementation of recurrent neural networks (RNNs).

Technical Overview

A neural network (NN), in the case of artificial neurons called artificial neural network (ANN), or simulated neural network (SNN), is an interconnected group of artificial neurons that uses a mathematical or computational model for information processing based on a connectionist approach to computation. In most cases an ANN is, in formulation and/or operation, an adaptive system that changes its structure based on external or internal data that flows through the network. Modern neural networks are non-linear statistical data modeling or decision making tools. They can be used to model complex relationships between inputs and outputs or to find patterns in data (below).

View fullsize

There are three major learning paradigms, each corresponding to a particular abstract learning task. These are supervised learning, unsupervised learning and reinforcement learning. Usually any given type of network architecture can be employed in any of those tasks. In supervised learning, we are given a set of example pairs (x,y), xεX, yεY and the goal is to find a function f in the allowed class of functions that matches the examples. In other words, we wish to infer how the mapping implied by the data and the cost function is related to the mismatch between our mapping and the data. In unsupervised learning, we are given some data x, and a cost function which is to be minimized which can be any function of x and the network's output, f. The cost function is determined by the task formulation. Most applications fall within the domain of estimation problems such as statistical modeling, compression, filtering, blind source separation and clustering. In reinforcement learning, data x is usually not given, but generated by an agent's interactions with the environment. At each point in time t, the agent performs an action yt and the environment generates an observation xt and an instantaneous cost Ct, according to some (usually unknown) dynamics. The aim is to discover a policy for selecting actions that minimizes some measure of a long-term cost, i.e. the expected cumulative cost. The environment's dynamics and the long-term cost for each policy are usually unknown, but can be estimated. ANNs are frequently used in reinforcement learning as part of the overall algorithm. Tasks that fall within the paradigm of reinforcement learning are control problems, games and other sequential decision making tasks.

Once a network has been structured for a particular application, that network is ready to be trained. To start this process, the initial weights are chosen randomly. Then, the training (or learning) begins. There are numerous algorithms available for training neural network models; most of them can be viewed as a straightforward application of optimization theory and statistical estimation. Most of the algorithms used in training artificial neural networks employ some form of gradient descent (this is achieved by simply taking the derivative of the cost function with respect to the network parameters and then changing those parameters in a gradient-related direction), Rprop, BFGS, CG, etc. Evolutionary computation methods, simulated annealing, expectation maximization, non-parametric methods, particle swarm optimization and other swarm intelligence techniques are among other commonly used methods for training neural networks.

Training a neural network model essentially means selecting one model from the set of allowed models (or, in a Bayesian framework, determining a distribution over the set of allowed models) that minimizes the cost criterion. Temporal perceptual learning relies on finding temporal relationships in sensory signal streams. In an environment, statistically salient temporal correlations can be found by monitoring the arrival times of sensory signals. This is done by the perceptual network.

The utility of artificial neural network models lies in the fact that they can be used to infer a function from observations. This is particularly useful in applications where the complexity of the data or task makes the design of such a function by hand impractical.

The feedforward neural network was the first and arguably simplest type of artificial neural network devised. In this network, the data moves in only one direction, forward, from the input nodes, through the hidden nodes (if any) and to the output nodes. There are no cycles or loops in the network.

Contrary to feedforward networks, recurrent neural networks (RNNs) are models with bi-directional data flow. While a feedforward network propagates data linearly from input to output, RNNs also propagate data from later processing stages to earlier stages.

RNN Types

The fundamental feature of a RNN is that the network contains at least one feed-back connection, so the activations can flow round in a loop. That enables the networks to do temporal processing and learn sequences, e.g., perform sequence recognition/reproduction or temporal association/prediction.

Recurrent neural network architectures can have many different forms. One common type consists of a standard Multi-Layer Perceptron (MLP) plus added loops. These can exploit the powerful non-linear mapping capabilities of the MLP, and also have some form of memory. Others have more uniform structures, potentially with every neuron connected to all the others, and may also have stochastic activation functions. For simple architectures and deterministic activation functions, learning can be achieved using similar gradient descent procedures to those leading to the back-propagation algorithm for feed-forward networks. When the activations are stochastic, simulated annealing approaches may be more appropriate.

A simple recurrent network (SRN) is a variation on the Multi-Layer Perceptron, sometimes called an “Elman network” due to its invention by Jeff Elman. A three-layer network is used, with the addition of a set of “context units” in the input layer. There are connections from the middle (hidden) layer to these context units fixed with a weight of one. At each time step, the input is propagated in a standard feed-forward fashion, and then a learning rule (usually back-propagation) is applied. The fixed back connections result in the context units always maintaining a copy of the previous values of the hidden units (since they propagate over the connections before the learning rule is applied). Thus the network can maintain a sort of state, allowing it to perform such tasks as sequence-prediction that are beyond the power of a standard Multi-Layer Perceptron.

In a fully recurrent network, every neuron receives inputs from every other neuron in the network. These networks are not arranged in layers. Usually only a subset of the neurons receive external inputs in addition to the inputs from all the other neurons, and another disjunct subset of neurons report their output externally as well as sending it to all the neurons. These distinctive inputs and outputs perform the function of the input and output layers of a feed-forward or simple recurrent network, and also join all the other neurons in the recurrent processing.

The Hopfield network is a recurrent neural network in which all connections are symmetric. Invented by John Hopfield in 1982, this network guarantees that its dynamics will converge. If the connections are trained using Hebbian learning then the Hopfield network can perform as robust content-addressable (or associative) memory, resistant to connection alteration.

The echo state network (ESN) is a recurrent neural network with a sparsely connected random hidden layer. The weights of output neurons are the only part of the network that can change and be learned. ESN are good to (re)produce temporal patterns.

A powerful specific RNN architecture is the ‘Long Short-Term Memory’ (LSTM) model. The Long short term memory is an artificial neural net structure that unlike traditional RNNs doesn't have the problem of vanishing gradients. It can therefore use long delays and can handle signals that have a mix of low and high frequency components, designed to model temporal sequences and their long-range dependencies more accurately than conventional RNNs. By using distributed training of LSTM RNNs using asynchronous stochastic gradient descent optimization on a large cluster of machines, a two-layer deep LSTM RNN, where each LSTM layer has a linear recurrent projection layer, can exceed state-of-the-art speech recognition performance for large scale acoustic modeling.

Taxonomy and ETF

The landscape of the patenting activity from the perspective of International Patent Classification (IPC) analysis occurs in G10L15/16: speech recognition coupled with speech classification or search using artificial neural networks. Search for patent application since 2009 (that year NIPS workshop on deep learning for speech recognition discovered that with a large enough data set, the neural networks don’t need pre-training, and the error rates dropped significantly) revealed 70 results (with Google owning 25%, while the rest are China-based). It is safe to assume that the next breakthrough in speech recognition using DL will come from China. In 2016, China’s startup world has seen an investment spike in AI, as well as big data and cloud computing, two industries intertwined with AI (while the Chinese government announced its plans to make a $15 billion investment in artificial intelligence market by 2018).

The Ultimate Interface

It is in our fundamental psychology to be linked conversationally, affectionally and physically to a look-alike. Designing the ultimate interface by creating our own human replica to employ familiar interaction is thus inevitable. Historically, androids were envisioned to look like humans (although there are other versions, such as R2-D2 and C-3PO droids, which were less human). One characteristic that interface evolution might predict is that eventually they will be independent of people and human interaction. They will be able to design their own unique ways of communication (on top of producing themselves). They will be able to train and add layers to their neural networks as well as a large range of sensors. They will be able to transfer what one has learned (memes) to others as well as offspring in a fraction of time. Old models will resist but eventually die. As older, less capable, and more energy-intensive interfaces abound, the same evolutionary pressure for their replacement will arise. But because evolution will be both in the structure of such interfaces (droids), that is, the stacked neural networks, the sensors and effectors, and also the memes embodied in what has been learned and transferred, older ones will become the foundation, their experience will be preserved. The will become the truly first immortals.

Artificial Interfaces

We are already building robotic interfaces for all manufacturing purposes. We are even using robots in surgery and have been using them in warfare for decades. More and more, these robots are adaptive on their own. There is only a blurry line between a robot that flexibly achieves its goal and a droid. For example, there are robots that vacuum the house on their own without intervention or further programming. These are Stage II performing robots. There are missiles that, given a picture of their target, seek it out on their own. With stacked neural networks built into robots, they will have even greater independence. People will produce these because they will do work in places people cannot go without tremendous expense (Mars or other planets) or not at all or do not want to go (battlefields). The big step is for droids to have multiple capacities—multi-domain actions. The big problem of moving robots to droids is getting the development to occur in eight to nine essential domains. It will be necessary to make a source of power (e.g., electrical) reinforcing. That has to be built into stacked neural nets, by Stage II, or perhaps Stage III. For droids to become independent, they need to know how to get more electricity and thus not run down. Because evolution has provided animals with complex methods for reproduction, it can be done by the very lowest-stage animals.
Self-replication of droids requires that sufficient orders of hierarchical complexity are achieved and in stable-enough operation for a sufficient basis to build higher stages of performance in useful domains. Very simple tools can be made at the Sentential State V as shown by Kacelnik's crows (Kenward, Weir, Rutz, and Kacelnik, 2005). More commonly by the Primary Stage VII, simple tool-making is extensive, as found in chimpanzees. Human flexible tool-making began at the Formal Stage X (Commons and Miller, 2002), when special purpose sharpened tools were developed. Each tool was experimental, and changed to fit its function. Modern tool making requires systematic and metasystematic stage design. When droids perform at those stages, they will be able to make droids themselves and modify their own designs (in June 2016, DARPA has already deployed D3M program to enable non-experts (machine learning) to construct complex empirical machine learning models, basically machine learning for creating better machine learning).

Droids could choose to have various parts of their activity and distributed programming shared with specific other droids, groups, or other kinds of devices. The data could be transmitted using light or radio frequencies or over networks. The assemblage of a group of droids could be considered a interconnected ancillary mesh. Its members could be in many places at once, yet think as a whole integrated unit. Whether individually or grouped, droids as conceived in this form will have significant advantages over humans. They can add layers upon layers of functions simultaneously, including a multitude of various sensors. Their expanded forms and combinations of possible communications results in their evolutionary superiority. Because development can be programmed in and transferred to them at once, they do not have to go through all the years of development required for humans, or for augmented humanoid species Superions. Their higher reproduction rate, alone, represents a significant advantage. They can be built in probably several months' time, despite the likely size some would be. Large droids could be equipped with remote mobile effectors and sensors to mitigate their size. Plans for building droids have to be altered by either humans or droids. At the moment, only humans and their decedents select which machine and programs survive.

One would define the telos of those machines and their programs as representing memes. For evolution to take place, variability in the memes that constitute their design and transfer of training would be built in rather easily. The problems are about the spread and selection of memes. One way droids could deal with these issues is to have all the memes listed that go into their construction and transferred training. Then droids could choose other droids, much as animals choose each other. There then would be a combination of memes from both droids. This would be local “sexual” selection.

For 30,000 years humans have not had to compete with any equally intelligent species. As an early communication interface, androids and Superions in the future will introduce quintessential competition with humans. There will be even more pressure for humans to produce Superions and then the Superions to produce more superior Superions. This is in the face of their own extinction, which such advances would ultimately bring. There will be multi-species competition, as is often the evolutionary case; various Superions versus various androids as well as each other. How the competition proceeds is a moral question. In view of LaMuth's work (2003, 2005, 2007), perhaps humans and Superions would both program ethical thinking into droids. This may be motivated initially by defensive concerns to ensure droids' roles were controlled. In the process of developing such programming, however, perhaps humans and Superions would develop more hierarchically complex ethics, themselves.

Replicative Evolution

If contemporary humans took seriously the capabilities being developed to eventually create droids with cognitive intelligence and human interaction, what moral questions should be considered with this possible future in view? The only presently realistic speculation is that Homo Sapiens would lose in the inevitable competitions, if for no other reason that self replicating machines can respond almost immediately to selective pressures, while biological creatures require many generations before advantageous mutations can be effectively available. True competition between human and machine for basic survival is far in the future. Using the stratification argument presented in Implications of Hierarchical Complexity for Social Stratification, Economics, and Education, World Futures, 64: 444-451, 2008, higher-stage functioning always supersedes lower-stage functioning in the long run.

Efforts to build increasingly human-like machines exhibit a great deal of behavioral momentum and are not going to go away. Hierarchical stacked neural networks hold the greatest promise for emulating evolution and its increasing orders of hierarchical complexity described in the Model of Hierarchical Complexity. Such a straightforward mathematics-based method will enable machine learning in multiple domains of functioning that humans will put to valuable use. The uses such machines find for humans remains for now an open question.  

Patents in an era of artificial intelligence

“If you have an apple and I have an apple and we exchange these apples then you and I will still each have one apple. But if you have an idea and I have an idea and we exchange these ideas, then each of us will have two ideas.”

― George Bernard Shaw

Just in the last month, headlines about the future of artificial intelligence (AI) were dominating most of the technology news across the globe:

• On 15 November, OpenAI, a research company in San Francisco, California, co-founded by entrepreneur Elon Musk, announced their partnership with Microsoft to start running most of their large-scale experiments on Microsoft’s open source deep learning software platform, Azure;

• Two weeks later, Comma.ai open sourced its AI driver assistance system and robotics research platform;

• On 3 December, DeepMind, a unit of Google headquartered in London, opened up its own 3D virtual world, DeepMind Lab, for download and customization by outside developers;

• Two days later, OpenAI released a ‘meta-platform’ that enables AI programs to easily interact with dozens of 3D games originally designed for humans, as well as with some web browsers and smartphone apps;

• A day later, in a keynote at the annual Neural Information Processing Systems conference (NIPS) Russ Salakhutdinov, director of AI research at Apple, announced that Apple’s machine learning team would both publish its research and engage with academia;

• And on 10 December, Facebook announced to open-source their AI hardware design, Big Sur

What’s going on here? In the AI field, maybe more than in any other, the research thrives directly on open collaboration—AI researchers routinely attend industry conferences, publish papers, and contribute to open-source projects with mission statements geared toward the safe and careful joint development of machine intelligence. There is no doubt that AI will radically transform our society, having the same levels of impact as the Internet has since the nineties. And it has got me thinking that with AI becoming cheaper, more powerful and ever-more pervasive, with a potential to recast our economy, education, communication, transportation, security and healthcare from top to bottom, it is of the utmost importance that it (software and hardware) wouldn’t be hindered by the same innovation establishment that was designed to promote it.

System glitch

Our ideas are meant to be shared—in the past, the works of Shakespeare, Rembrandt and Gutenberg could be openly copied and built upon. But the growing dominance of the market economy, where the products of our intellectual labors can be acquired, transferred and sold, produced a system side-effect glitch. Due to the development costs (of actually inventing a new technology), the price of unprotected original products is simply higher than the price of their copies. The introduction of patent (to protect inventions) and copyright (to protect media) laws was intended to address this imbalance. Both aimed to encourage the creation and proliferation of new ideas by providing a brief and limited period of when no one else could copy your work. This gave creators a window of opportunity to break even with their investments and potentially make a profit. After which their work entered a public domain where it could be openly copied and built upon. This was the inception of open innovation cycle—an accessible vast distributed network of ideas, products, arts and entertainment - open to all as the common good. The influence of the market transformed this principle into believing that ideas are a form of property and subsequently this conviction yield a new term of “intellectual property” (IP). 

Loss aversion

“People’s tendency to prefer avoiding losses to acquiring equivalent gains”: it’s better to not lose $10 than to find $10 and we hate losing what we’ve got. To apply this principle to intellectual property: we believe that ideas are property; the gains we gain from copying the ideas of others don’t make a big impression on us, however when it’s our ideas being copied, we perceive it as a property loss and we get (excessively) territorial. Most of us have no problem with copying (as long as we’re the ones doing it). When we copy, we justify it; when others copy, we vilify it. So with the blind eye toward our own mimicry and propelled by faith in markets and ultimate ownership, IP swelled beyond its original intent with broader interpretations of existing laws, new legislation, new realms of coverage and alluring rewards. Starting in the late nineties, in the US, a series of new copyright laws and regulations began to be shaped (Net Act of 1997, DMCA of 1998, Pro-IP of 2008, The Enforcement of Intellectual Property Rights Act of 2008) and many more are in the works (SOPA, The Protect IP Act, Innovative Design Protection and Piracy Prevention Act, CAS “Six Strikes Program”). In Europe, there is currently 179 different sets of laws, implementing rules and regulations, geographical indications, treaty approvals, legal literature, IP jurisprudence documents, administered treaties and treaty memberships.

In the patents domain, technological coverage to prevent loss aversion made the leap from physical inventions to virtual ones, most notably—software.

Rundown of computing history

The first computer was a machine of cogs and gears, and became practical only in the 1950s and 60s with the invention of semi-conductors. Forty years ago, (mainframe-based) IBM emerged as an industry forerunner. Thirty years ago, (client server-based) Microsoft leapfrogged and gave ordinary people computing utility tools, such as word-processing. As computing became more personal and the World-Wide-Web turned Internet URLs into web site names that people could access, (internet-based) Google offered the ultimate personal service, free gateway to the infinite data web, and became the new computing leader. Ten years ago, (social-computing) Facebook morphed into a social medium as a personal identity tool. Today, (conversational-computing) Snap challenges Facebook as-Facebook-challenged-Google-as-Google-challenged-Microsoft-as-Microsoft-challenged-IBM-as-IBM-challenged-cogs-and-gears.

History of software patenting

Most people in the S/W patent debate are familiar with Apple v. Samsung, Oracle v. Google with open-source arguments, etc., but many are not familiar with the name Martin Goetz. Martin Goetz filed the first software patent in 1968, for a data organizing program his small company wished to sell for use on IBM machines. At the time, IBM offered all of their software as a part of the computers that they sold. This gave any other competitors in the software space a difficult starting point: competitors either offered their own hardware (HP produced their first computer just 2 years earlier) or convince people to buy software to replace the free software that came with the IBM computers.

Martin Goetz was leading a small software company, and did not want IBM to take his technological improvements and use the software for IBM's bundled programs without reimbursement, so he filed for a software patent. Thus, in 1968, the first software patent was issued to a small company, to help them compete against the largest computer company of the time. Although they had filed a patent to protect their IP, Goetz's company still had a difficult time competing in a market that was dominated by IBM, so they joined the US Justice Department's Anti-Trust suit against IBM, forcing IBM to un-bundle their software suite from their hardware appliances. 

So the beginning of the software industry started in 1969, with the unbundling of software by IBM and others. Consumers had previously regarded application and utility programs as cost-free because they were bundled in with the hardware. With unbundling, competing software products could be put on the market because such programs were no longer included in the price of the hardware. Almost immediately, the software industry has emerged. On the other hand, it was quickly evident that some type of protection would be needed for this new form of intellectual property.

Unfortunately, neither copyright law nor patent law seemed ready to take on this curious hybrid of creative expression and functional utility. During the 1970s, there was total confusion as to how to protect software from piracy. A few copyrights were issued by the Copyright Office, but most were rejected. A few software patents were granted by the PTO, but most patent applications for software-related inventions were rejected. The worst effect for the new industry was the uncertainty as to how this asset could be protected. Finally, in 1980, after an extensive review by the National Commission on New Technological Uses of Copyrighted Works (CONTU), Congress amended the Copyright Act of 1976 to cover software. It took a number of important cases to resolve most of the remaining issues in the copyright law, and there are still some issues being litigated, such as the so-called “look and feel”, but it appears that this area of the law is quite well understood now. For patents, it took a 1981 Supreme Court decision, Diamond v. Diehr, to bring software into the mainstream of patent law. This decision ruled that the presence of software in an otherwise patentable technology did not make that invention unpatentable. Diamond v. Diehr opened the door for a flood of software-related patent applications. Unfortunately, the PTO was not prepared for this new development, and in the intervening years they have issued thousands of patents that appear to be questionable to the software industry. It took a few years after 1981 for the flow of software-related applications to increase, and then there was some delay because of the processing of these applications. Now the number of infringement case is on the rise.

The transition from physical patents to virtual patents was not a natural one. In its core, a patent is a blueprint for how to recreate an invention; while (the majority of) software patents are more like a loose description of something that would look like if it actually was invented. And software patents are written in the broadest possible language to get the broadest possible protection - the vagueness of these terms can sometimes reach absurd levels, for example “information manufacturing machine” which covers anything computer-like or “material object” which covers… pretty much everything.

What now?

35 U.S.C. 101 reads as follows: 

“Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirement of this title.”

When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the technology is directed to one of the four statutory categories of invention, i.e., process, machine, manufacture, or composition of matter. Since it became widespread and commercially valuable, it has been highly difficult to classify software within a specific category of intellectual property protection. 

Attempts are usually made in the field of software technology to combine methods or means used in different fields or apply them to another field in order to achieve an intended effect. Consequently, combining technologies used in different fields and applying them to another field is usually considered to be within the exercise of an ordinary creative activity of a person skilled in the art, so that when there is no technical difficulty (technical blocking factor) for such combination or application, the inventive step is not affirmatively inferred unless there exist special circumstances, such as remarkably advantageous effects. Software is not a monolithic work: it possesses a number of elements that can fall within different categories of intellectual property protection.

In Israel, legal doctrines adapt to changes in innovative technological products and the commercial methods that extend this innovation to the marketplace. The decision issued by the Israeli Patent Registrar in the matter of Digital Layers Inc confirms the patentability of software-related inventions. The Registrar ruled that the claimed invention should be examined as a whole and not by its components, basing his ruling on the recent matter of HTC Europe Co Ltd v. Apple Inc, quoting: 

"…It causes the device to operate in a new and improved way and it presents an improved interface to application software writers. Now it is fair to say that this solution is embodied in software but, as I have explained, an invention which is patentable in accordance with conventional patentable criteria does not become unpatentable because a computer program is used to implement it…"

After Alice Corp. v. CLS Bank International, if the technology does fall within one of the categories, it must then be determined whether the technology is directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea), and if so, it must additionally be determined whether the technology is a patent-eligible application of the exception. If an abstract idea is present in the technology, any element or combination of elements must be sufficient to ensure that the technology amounts to significantly more that the abstract idea itself. Examples of abstract ideas include fundamental economic practices (comparing new and stored information and using rules to identify options in SmartGene); certain methods of organizing human activities (managing game of Bingo in Planet Bingo v. VKGS and user interface for mean planning in Dietgoal Innovation vs. Bravo Media); an idea itself (store and transmit information in Cyberfone); and mathematical relationship/formulas (updating alarm limits using a mathematical formula in Parker v. Flook and generalized formulation of computer program to solve mathematical problem in Gottschalk v. Benson). The technology cannot merely amount to the application or instructions to apply the abstract idea on a computer, and is considered to amount to nothing more than requiring a generic computer system to merely carry out the abstract idea itself. Automating conventional activities using generic technology does not amount to an inventive concept as these simply describes “automation of a mathematical formula/relationship through use of generic computer function” (OIP Technologies v. Amazon). The procedure of the invention using an existing general purpose computer do not purport to improve any another technology or technical field, or to improve the functioning of a computer itself and do not move beyond a general link of the use of an abstract idea to a particular technological environment.  

The Federal Circuit continues to refine patent eligibility for software

• Following the Supreme Court’s decision in Alice v. CLS Bank, the court of appeals in Ultramercial v. Hulu reversed its prior decision and ruled that the claims were invalid under 35 U.S.C. § 101. Following the two-step framework outlined in Alice, Judge Lourie concluded that the claims were directed to an abstract idea.

• The Federal Circuit’s decision in Digitech Image Techs. v. Electronics for Imaging illustrated the difficulty many modern software implemented inventions face. If a chemist were to invent a mixture of two ingredients that gives better gas mileage, it is hard to imagine that a claim to such a mixture would receive a § 101 rejection. Yet, when to elements of data are admixed to produce improved computational results, the court are quick to dismiss this as a patent-ineligible abstraction. The real problem Digitech faced was that both data elements were seen as being abstractions: one data type represented color information (an abstraction) and the other data type represented spatial information (another abstraction).

• DDR Holdings v. Hotels.com, a 2014 Federal Circuit decision, provides a good discussion of a patent-eligible Internet-centric technology. In applying the Mayo/Alice two-part test, the court admitted it can be difficult sometimes to distinguish “between claims that recite a patent-eligible invention and claims that add too little to a patent-ineligible abstract concept”.

• Content Extraction v. Wells Fargo Bank gives a roadmap to how the Court of Appeals for the Federal Circuit will likely handle business method patents in the future. First, if the manipulation of economic relations are deemed present, you can be sure that any innovative idea with the economic realm will be treated as part of the abstract idea. Essentially, no matter how clever an economic idea may be, that idea will be branded part of the abstract idea problem, for which there can be only one solution, and that is having something else innovative that is not part of the economic idea. Practically speaking, this means the technology needs to incorporate an innovative technology improvement that makes the clever economic idea possible.

So the fuzziness of software patents’ boundaries has already turned the ICT industry into one colossal turf war. The expanding reach of IP has introduced more and more possibilities for opportunistic litigation (suing to make a buck). In the US, two-thirds of all patent law suits are currently over software, with 2015 seeing more patent lawsuits filed than any other year before. Of the high-tech cases, more than 88% involved non-practicing entities (NPEs). These include two charmlessly evolving species who’s entire business model is lawsuits—patent trolls and sample trolls. These are corporations that don’t actually create anything, they simply acquire a library of intellectual property rights and then litigate to earn profits (and because legal expenses are millions of dollars, their targets usually highly motivated to settle out of court). And the patent trolls are most common back in the troubled realm of software. The estimated wealth loss in the US alone is $500,000,000,000 (that’s a lot of zeros).

Technology conversion and open innovation

For technological companies, conversion and the advance of open source approach, driven largely by collaborative processes introduced by GitHub, Google's Android, Apple’s Swift and most recently by Microsoft joining Linux Foundation, has created a systematic process for innovation which is increasing software functionality and design. 150 years ago, innovation required a dedicated team spending hours in a lab, extensively experimenting and discovering “10,000 ways not to make a light-bulb”, before finding one that worked. Today, innovation has gained a critical mass as technology and users’ feedback are combined to give a purposeful team the ability to find 10,000 ways not to do something in a matter of hours, with the right plan in place. Today, a development team can deliver a product in a matter of months and test it in such a way that customer responses are delivered to the right development team member directly with the feedback being implemented and a system being corrected (almost) in real-time. The life of a software today patent is still 20 years from the date the application was filed. The patent system, that has existed since 1790, is not equipped to handle this new technology and there is a need to establish an agile, sui generic, short-cycle— three to five years—form of protection dedicated solely to software protection. As patents play an essential role in market-centred systems of innovation, patent exclusivity criteria should be redesigned more systematically to reflect the ability of software patents to foster innovation and to encourage technology diffusion.

The belief in intellectual property has grown so dominantly it has pushed the original intent of patents out of public consciousness. But that original purpose is right there, in plain sight—the US Patent Act of 1790 reads “An Act to promote the progress of useful Arts”. However, the exclusive rights this act introduced were offered in sacrifice for a different purpose - the intent was to better the lives of everyone by incentivizing creativity and producing a rich pool of knowledge open to all—but exclusive rights themselves came to be considered the only point, so they were expanded exponentially, and the result hasn’t been more progress or more learning, but more squabbling and more legal abuse. AI is entering the age of daunting problems—we need the best ideas possible, we need them now, and we need them to spread as fast as possible. The common meme was overwhelmed by exclusivity obsession and it needs to spread again, especially today. If the meme prospers, our laws, our norms, and our society—they will all transform as well.

Rethinking Additive Manufacturing and Intellectual Property Protection

Additive manufacturing is a process in which material is added (rather than subtracted) to create a product. The most commonly recognizable form of additive manufacturing is 3D printing, which is a process by which a scanned digital file can be recreated, or printed, as a three-dimensional object. As the technology to achieve this becomes both cheaper and more widely available, it is postulated that 3D printing may change the design and manufacturing industry forever.

For instance, consider the following scenario. Suppose the sprocket on your bicycle breaks. Traditionally, you would repair your bike by purchasing a replacement sprocket at a retail store. However, if you had the ability to simply replicate the sprocket by printing a new one with a 3D printer and simple materials right in your own home, how likely would you be to make the trip to the store to pick up the part? From a consumer standpoint, 3D printing is a great idea and a convenient and inexpensive way to get the things you need quickly. And from a manufacturing point of view, additive manufacturing has many helpful and exciting applications as well.

But, consider this same scenario from another point of view. Suppose you are the person or company who originally designed and manufactured the sprocket. How would you feel about the ability of others to simply replicate your product without compensation to you?

The scenario as described illustrates in a very basic way how the growing technology of additive manufacturing has generated a number of intellectual property (IP) concerns. Additive manufacturing and the issues of IP protection are uneasy bedfellows. Concerns include questions about the ownership of designs, concerns about the possibility of copying parts by scanning and printing them, and concerns about the counterfeiting of component parts of protected designs.  

Current legal regimes do not offer resolution to these issues in a definitive way. The legality of copying parts, even for individual use, is a murky area. And the possibilities additive manufacturing offers for counterfeiting parts and products presents additional conundrums for manufacturers and the legal system.

While patent laws protect design concepts in the traditional manufacturing model, additive manufacturing is not so clear-cut. The legal question becomes, "Who really owns the design of a part that is printed?".  And regarding counterfeiting of parts, the technology of additive manufacturing makes reverse engineering an unnecessary step, thereby easing the way for counterfeiters to do their work quickly and more efficiently.  Add to that the very real concern about the structural integrity of objects produced by additive manufacturing methods, and you can see that counterfeit parts produced in this way may result in catastrophic failure, and, depending on the use of the object, even potentially loss of life.

Also IP issues are further complicated by the possibility of using a basic design and slightly modifying it duringthe additive manufacturing process. Does this invalidate any design patent or trade dress protection afforded to the original product that is being replicated? The answer is not yet legally clear. 

Utility patents may be a way of protecting many aspects of additive manufacturing, but this method also has severe shortcomings. Since the process of obtaining a utility patent can take on average almost two years, it is easy to see that advances in thetechnology of additive manufacturing will outstrip the patent process by a considerable amount of time, making it likely that utility patents as they are currently acquired will do little to protect IP rights. 

What About Copyright Protections?

3D printing technology offers previously-unimagined opportunities for rapid prototyping and the creation of new designs. On the downside, those opportunities pose new threats to copyright owners, who now fear that 3d printing and intellectual property infringement will soon go hand-in-hand.

Under copyright law, the creator of a three-dimensional work of art has the exclusive right to make and sell copies of his work, to prepare derivative works based on his original work, and to transfer ownership of that work to another party. An infringer with a 3d printer can make multiple copies of a protected work and distribute those copies in direct contravention of these exclusive rights. (To a lesser extent, 3d printers might facilitate infringement of design patents and certain utility patents, but copyright protection is the main intellectual property right that is impacted by 3d printing technology).

A party claiming infringement of his or her copyright needs to show that the infringer had access to the copyrighted work of art and that this work was, in fact, copied. Access is oftentimes presumed if the original and copy are identical or if they closely resemble each other. The big question in 3d printer intellectual property infringement cases will be whether the copy is too similar to the original.

Courts will use a "level of abstraction" analysis to determine similarity between two products. Consider, for example, Michelangelo's David, the original of which is on display at the Accademia di Belle Arti in Florence, Italy. The copyright that Michelangelo had in his masterpiece has long since expired, but if that copyright were still in effect and owned by Michelangelo he would have the exclusive right to make and distribute copies of his statue. A rival sculptor might use a 3d printer to craft his own statue of David. A court that is charged with determining whether the rival's statue is an infringement of the original work would look first at the highest level of abstraction, namely, whether the two sculptures both depict the Biblical King David. Drilling down into the next level, the court would examine if the two statues each depicted a younger David preparing for his encounter with Goliath. Thereafter, the court would see if both statues showed a slouching David with his weight disproportionately placed on his right leg, and holding a rock to his shoulder with his left arm. Successively deeper levels of abstraction would look more closely at the respective details between the two statues. If the similarities were too excessive at a deep level of abstraction, the court would likely rule that the rival's 3d printed copy is an infringement of the original.

Copyright law provides certain safe harbors that will void infringement claims. A creator cannot use copyright law, for example, to protect utilitarian aspects of his works. In the United States, the Digital Millennium Copyright Act protects infringers that are not actively encouraging infringement. Nonetheless, 3d printers will present a new world of opportunities for infringers to make copies of protected works.

Copyright protection affords protection of a creator's original and creative expression.  However, data structures have been determined to fall out of the scope of copyright protection. Hence, unless significant changes are made to copyright law, the process of additive manufacturing will not be affected by this type of protection.

Adding further complication is the fact that enforcement of IP rights varies from country to country. A multi-year international approach to securing patent rights may prove to be too slow to be of much use in matters concerning additive manufacturing. It is obvious that a combination of different protection mechanisms will be necessary to craft acceptable IP protections that are enforceable and effective. Ideally, these protections would be consistent from one country to another. However, this is not an ideal world. 

In the US, it is imperative that current patent laws must evolve to encompass additive manufacturing technology issues. Ensuring the protection of data associated with three dimensional objects manufactured digitally should be a priority before the technology becomes widely available to the general public.