Mark Ring

IJCAI Conference 2013 Conference Paper

Better Generalization with Forecasts

• Tom Schaul
• Mark Ring

IJCAI Conference 2011 Conference Paper

Q-Error as a Selection Mechanism in Modular Reinforcement-Learning Systems

• Mark Ring
• Tom Schaul

This paper introduces a novel multi-modular method for reinforcement learning. A multi-modular system is one that partitions the learning task among a set of experts (modules), where each expert is incapable of solving the entire task by itself. There are many advantages to splitting up large tasks in this way, but existing methods face difficulties when choosing which module(s) should contribute to the agent's actions at any particular moment. We introduce a novel selection mechanism where every module, besides calculating a set of action values, also estimates its own error for the current input. The selection mechanism combines each module's estimate of long-term reward and self-error to produce a score by which the next module is chosen. As a result, the modules can use their resources effectively and efficiently divide up the task. The system is shown to learn complex tasks even when the individual modules use only linear function approximators.

NeurIPS Conference 1997 Conference Paper

RCC Cannot Compute Certain FSA, Even with Arbitrary Transfer Functions

• Mark Ring

Existing proofs demonstrating the computational limitations of Re(cid: 173) current Cascade Correlation and similar networks (Fahlman, 1991; Bachrach, 1988; Mozer, 1988) explicitly limit their results to units having sigmoidal or hard-threshold transfer functions (Giles et aI. , 1995; and Kremer, 1996). The proof given here shows that for any finite, discrete transfer function used by the units of an RCC network, there are finite-state automata (FSA) that the network cannot model, no matter how many units are used. The proof also applies to continuous transfer functions with a finite number of fixed-points, such as sigmoid and radial-basis functions.

NeurIPS Conference 1992 Conference Paper

Learning Sequential Tasks by Incrementally Adding Higher Orders

• Mark Ring

An incremental, higher-order, non-recurrent network combines two properties found to be useful for learning sequential tasks: higher(cid: 173) order connections and incremental introduction of new units. The network adds higher orders when needed by adding new units that dynamically modify connection weights. Since the new units mod(cid: 173) ify the weights at the next time-step with information from the previous step, temporal tasks can be learned without the use of feedback, thereby greatly simplifying training. Furthermore, a the(cid: 173) oretically unlimited number of units can be added to reach into the arbitrarily distant past. Experiments with the Reber gram(cid: 173) mar have demonstrated speedups of two orders of magnitude over recurrent networks.