Arrow Research search

Author name cluster

Waleed Meleis

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers
2 author rows

Possible papers

5

AAMAS Conference 2017 Conference Paper

Dynamic Generalization Kanerva Coding in Reinforcement Learning for TCP Congestion Control Design

  • Wei Li
  • Fan Zhou
  • Waleed Meleis
  • Kaushik Chowdhury

Traditional reinforcement learning (RL) techniques often encounter limitations when solving large or continuous stateaction spaces. Training times needed to explore the very large space are impractically long, and it can be difficult to generalize learned knowledge. A compact representation of the state space is usually generated to solve both problems. However, simple state abstraction often cannot achieve the desired learning quality, while expert state representations usually involve costly hand-crafted strategies. We propose a new technique, generalization-based Kanerva coding, that automatically generates and optimizes state abstractions for learning. When applied to adapting the congestion window of the highly complex TCP congestion control protocol, a standard Internet protocol, this technique outperforms the current standard-TCP New Reno by 59. 5% in throughput and 6. 5% in delay. Our technique also achieves a 35. 2% improvement in throughput over the best previously proposed Kanerva coding technique when applied in the same context.

PDF

AAMAS Conference 2010 Conference Paper

Spectrum Management of Cognitive Radio Using Multi-agent Reinforcement Learning

  • Cheng Wu
  • Kaushik Chowdhury
  • Marco Di Felice
  • Waleed Meleis

Wireless cognitive radio (CR) is a newly emerging paradigmthat attempts to opportunistically transmit in licensed frequencies, without affecting the pre-assigned users of thesebands. To enable this functionality, such a radio must predict its operational parameters, such as transmit power andspectrum. These tasks, collectively called spectrum management, is difficult to achieve in a dynamic distributed environment, in which CR users may only take local decisions, andreact to the environmental changes. In this paper, we introduce a multi-agent reinforcement learning approach basedspectrum management. Our approach uses value functionsto evaluate the desirability of choosing different transmission parameters, and enables efficient assignment of spectrums and transmit powers by maximizing long-term reward. We then investigate various real-world scenarios, andcompare the communication performance using different setsof learning parameters. We also apply Kanerva-based function approximation to improve our approach's ability to handle large cognitive radio networks and evaluate its effect oncommunication performance. We conclude that our reinforcement learning based spectrum management can significantly reduce the interference to the licensed users, whilemaintaining a high probability of successful transmissions ina cognitive radio ad hoc network.

PDF

AAMAS Conference 2008 Conference Paper

Adaptive Kanerva-based Function Approximation for Multi-Agent Systems

  • Cheng Wu
  • Waleed Meleis

In this paper, we show how adaptive prototype optimization can be used to improve the performance of function approximation based on Kanerva Coding when solving largescale instances of classic multi-agent problems. We apply our techniques to the predator-prey pursuit problem. We first demonstrate that Kanerva Coding applied within a reinforcement learner does not give good results. We then describe our new adaptive Kanerva-based function approximation algorithm, based on prototype deletion and generation. We show that probabilistic prototype deletion with random prototype generation increases the fraction of test instances that are solved from 45% to 90%, and that prototype splitting increases that fraction to 94%. We also show that optimizing prototypes reduces the number of prototypes, and therefore the number of features, needed to achieve a 90% solution rate by up to 87%. These results demonstrate that our approach can dramatically improve the quality of the results obtained and reduce the number of prototypes required. We conclude that adaptive prototype optimization can greatly improve a Kanerva-based reinforcement learner’s ability to solve large-scale multi-agent problems.

PDF