Status-based Routing in Baggage Handling Systems: Searching Verses Learning

of 12
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information Report
Category:

Pets & Animals

Published:

Views: 6 | Pages: 12

Extension: PDF | Download: 0

Share
Description
Status-based Routing in Baggage Handling Systems: Searching Verses Learning
Tags
Transcript
  IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 40, NO. 2, MARCH 2010 189 Status-based Routing in Baggage Handling Systems:Searching Verses Learning Michael Johnstone, Doug Creighton, and Saeid Nahavandi  , Senior Member, IEEE   Abstract —This study contributes to work in baggage handlingsystem (BHS) control, specifically dynamic bag routing. Althoughstudies in BHS agent-based control have examined the need forintelligent control, but there has not been an effort to explore thedynamic routing problem. As such, this study provides additionalinsight into how agents can learn to route in a BHS. This study de-scribesaBHSstatus-basedroutingalgorithmthatapplieslearningmethodstoselectcriteriabasedonroutingdecisions.Althoughnu-merous studies have identified the need for dynamic routing, littleanalytic attention has been paid to intelligent agents for learningrouting tables rather than manual creation of routing rules. Weaddress this issue by demonstrating the ability of agents to learnhow to route based on bag status, a robust method that is able tofunction in a variety of different BHS designs.  IndexTerms —Airportoperations,materialshandling,reinforce-ment learning (RL), search methods. I. I NTRODUCTION T HE PRIMARY goal of a baggage handling system (BHS),as with any material handling system (MHS), is to transferitems from system inputs to system outputs. A major factor thatmakesaBHSaninterestingareaforstudyistheenvironmentthata BHS operates in. The volume of bags entering the BHS, themultitude of different aircraft capacities, changing flight sched-ules, lost bags, barcode misreads, early bags, late bags, andequipment downtime, all combine to make a highly stochas-tic and dynamic environment. Increased security requirementsnecessitate screening of all bags before being loaded onto air-craft[1],[2],increasingdemandsonaBHS,andmakingcontrolstrategies, is all the more challenging. Compounding the prob-lem of 100% checked baggage screening is an increase of 30%incheckedbaggage,duetothebanningliquidsandgelsincarry-on baggage [3], after British police thwarted a terrorist attemptto blowup aircraft travelling to the U.S.To explore these challenges a detailed overview of a BHSwill be given, followed by a review of previous paper relating toconveyor systems and their applicability to a BHS. This sectionsets the scene for the problem, while subsequent sections of this paper draw on analysis methods for similar systems used inthe past, finally concluding with an approach most suited to theBHS environment, whereby better throughput and security can ManuscriptreceivedAugust4,2008;revisedMarch17,2009.FirstpublishedDecember 22, 2009; current version published February 18, 2010. This work was funded by the Australian Research Council. This paper was recommendedby Associate Editor R. W. Brennan.The authors are with the Center for Intelligent System Research, Deakin Uni-versity, Waurn Ponds, Vic. 3217, Australia (e-mail: mjohnsto@deakin.edu.au;dougc@deakin.edu.au; nahavand@deakin.edu.au).Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.Digital Object Identifier 10.1109/TSMCC.2009.2035519 be more readily achieved, using new approaches to control thecomplex system.II. B ACKGROUND This section will look into all aspects of a BHS, includingthe primary method to convey bags, the operation, the flow,the environment, and metrics used for analysis. This provides aknowledge base before a review of related work is undertakenand the control requirements for a BHS is described.  A. BHS Overview1) Transport:  A BHS can use a variety of methods to con-vey bags. Belt conveyors, totes, tilt trays, or destination-codedvehicles (DCV) can be used exclusively or in combination, totransport bags from check in to departure gates.InaDCVsystemindividualvehiclescanindependentlymovealong a network of rails, effectively connecting every input toevery output, providing a highly connected system. As DCVstravel much faster than conveyors they are useful in situations,where long distances must be covered between check in anddeparture gate.Tilt tray and tote systems differ to DCV systems as a fixedpathisfollowed,unliketheindependentDCVs.Tilttraysystemsarenormallyusedforsortation,withbeltconveyorstransportingbags to the sorting system. Unlike tilt trays, tote systems can beused from check in to departure.Conveyor-based systems convey bags along the belts of theconveyors. These systems can operate independently, transport-ingbagsfrominputstoexits,orcanbeintegratedwithasortationsystem.These systemscan be quick to installand can be reused,while other systems are more complex to control and are notreadily adjustable [4].This research focuses on conveyor-based systems, where thecontrol techniques developed can be generically applied to anyother conveyor-based systems, such as those in a cargo facility,a warehouse, or the many other situations that makes use of aMHS. 2) BHS Operation:  There are many requirements, someeven competing that makeup the environment in which a BHSoperates. The fundamental operations can be summarized asscanning, screening, and delivery, in other words, identifyingbags with flight information is known, ensuring bags are safe toload onto the aircraft, and transportation of bags to the correctsystem output. These primary operations are discussed later. a) Scanning:  Scanning is the process of identifying bagsastheyflowthroughthesystem.Withoutscanningbagsitwouldnot be possible to differentiate among the thousands of bags 1094-6977/$26.00 © 2009 IEEE Authorized licensed use limited to: DEAKIN UNIVERSITY LIBRARY. Downloaded on April 16,2010 at 06:11:46 UTC from IEEE Xplore. Restrictions apply.  190 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 40, NO. 2, MARCH 2010 entering the system and automated delivery to the correct de-parture gate could not occur. To differentiate among bags, eachis assigned a unique ID using one of two possible methods,either barcode tags or radio frequency tags.Barcode tags are the most common format of the two. Theten-digit ID printed on a tag is used to match the bag with adatabase entry, specifying flight and passenger details. A tagis attached to bags at check in and is placed strategically toallow a barcode scanner array to scan the barcode as the bagtravels through the BHS. Automatic tag readers (ATR) are usedto provide an automated scan, operating at a success rate of 75–90% for locally checked in bags [6]. Failed scans must besent to operators for manual scanning.Radio frequency identification (RFID) has been proposed asa solution to resolve the errors associated with barcode tags [7].RFID operates by incorporating a chip that emits a radio signaldetected by an antenna, eliminating the line-of-site required bybarcode scanners. This solution operates at 95–99% accuracy,reducing the volume of mishandled bags, but at an increasedcost. b) Screening:  Screening is the act of ensuring a bag issafe to be loaded onto the aircraft. Screening of checked bag-gage ideally occurs inline in the BHS, while carry-on baggageand passengers themselves are screened between check in andboarding.Inline screening can be performed in several ways [8]. Inlinescreening causes a bottleneck within the system as bag speedsmust be reduced to perform a thorough inspection. To alleviatethe bottleneck, parallel lines are used to increase throughput.Another method is to layer the screening process, using high-speed explosive detection systems (EDS) in the first layer toinspect all bags. Subsequent layers manage bags that are unabletobeclearedandcanbeacombinationofoperatorimagereview,manualinspection,explosivetracedetection(ETD),orasecond,more detailed EDS. c) Delivery:  Only after a bag passes screening and hasbeen identified through scanning, can be delivered to its exitpoint. These exit points are assigned to a flight for a period of time before the flights departure time (STD).A sortation process is commonly used to sort bags based ontheir flight, once the bag has been scanned and screened. High-speed tilt tray loops are suitable in high-volume systems, whilelow-volume systems are adequately serviced by conveyors. 3) BHS Flow:  The flow of bags through a BHS varies fromsystemtosystemwithnosetorderbetweenscanningandscreen-ing processes, obviously delivery to the exit point is always last.There are many variations that exist to determine bag flow, theplacement of equipment to manage scanning and screening, thenumber and capacity of EDS, the number of inputs and outputs,and the number of conveyor lines, just to name a few. This widevariety calls for control and analysis methods robust enough tomanage any combination that is presented. 4) Flow Analysis:  It is essential to measure how effectivelya BHS is performing its primary goals, scanning, screening,and delivery, as it is influenced by its operating environment.BHS not only deliver security checked bags to their correctdestinations, bur also complete the task within a certain timeframe, sharing the load across system resources. The BHS QoSconcept has been defined as the percentage of bags delivered ontime to the correct location combined with system availability[9]. This concept can be extended to consider in-system times,throughputs, screening and scanning rates, and other factorsused to measure the operation of a modern BHS. To measureQoS, two methods have been identified in literature, flow rateand in-system time [10]. Flow rates deal with the capacity of the system, or conveyor lines within the system. The travel orin-system time is a measure of how long bags spend in the BHSfrom entry to exit. Ideally, a BHS wishes to have a maximizedthroughput and a minimized in-system time as these two factorscombined mean that a BHS will be able to accommodate moreflights [9], [11], [12], making it more commercially viable [13].  B. Related Work  The traditional method to control a BHS has focused on di-recting bags along a prior computed shortest path [13]. Thesepaths are static, they will always be used by the control systemregardless of the system state. The problems with static shortestpath routing are often described in literature, the major pointbeing that they do not adapt to changes in traffic flow, whereasdynamic routing will adapt and provide better performance.An early attempt to break from the traditional methods of BHS control met with disaster in the Denver International Air-port [14]. Here, the control system was to manage 4000 DCVsalong 33 km of track as individual vehicles were fed by a 9 kmof conveyor network. Highly visible problems of bag jams, mu-tilations, and misalignments were quickly evident, and deeperproblems such as line balancing and empty carrier managementwere observed when investigation into delivery times were notbeing met. The complexity of the control system caused a 16-month delay to the opening of the airport and a secondary, moreconventional BHS, was installed in parallel as a solution to theproblem.Siesennop  et al.  [10] used simulation to analyze the con-trol logic for a DCV-based BHS, while their work focused onthe management of empty DCVs, especially to prevent queuesbuilding at inputs, in general, they made valid observationsabout MHS. First in terms of analysis, flow rates and traveltimes are a major factor to consider when looking at the per-formance of the system. Secondly, issues around the control of DCVs mirror similar problems found in routing protocols usedin computer networks, excessive flows along certain links andsystem traffic imbalances. These types of problems can alsooccur in a conveyor-based system if the conveyor lines are rununbalanced.Fay  et al.  [9] investigated a decentralized control strategy fora DCV segment within a BHS. Their argument was that while acentral control strategy could potentially find an optimal controlstrategy in terms of minimum travel time or optimal QoS, thesestrategies can suffer due to the existence of a single point of failure. The decentralized strategy proposed was market basedto assign empty DCVs to bags. They found their market-basedapproach to reduce the in-system time of bags compared to acentralized first in first out (FIFO) rule. The second aspect to Authorized licensed use limited to: DEAKIN UNIVERSITY LIBRARY. Downloaded on April 16,2010 at 06:11:46 UTC from IEEE Xplore. Restrictions apply.  JOHNSTONE  et al. : STATUS-BASED ROUTING IN BAGGAGE HANDLING SYSTEMS: SEARCHING VERSES LEARNING 191 the paper addressed routing DCVs through a rail network. Theauthors recommend using a routing protocol, similar to a link-state protocol found on the Internet, to efficiently route DCVs,as other methods in literature do not scale to accommodatevehicles numbering in the thousands. Details on the benefits of this method are presented, but implementation details sketchilypoint toward routing decisions based on congestion. The BHSchosenforsimulationtotestthesestrategiesappearssosimpleasto not require advanced routing control and there are no resultsgiven for the routing strategy. The author’s conclusion that amarket-based approach to DCV assignment and Internet likerouting strategies look promising, is valid, however, insufficientinformationsurroundingtheroutingstrategywaspresented.Theidea that routing control strategies for a MHS can be influencedby other areas, i.e., the Internet, is a good one and will bedeveloped further in this research and is found in other authorswork.In another decentralized approach to control of a BHS, thistime a conveyor-based BHS [11] agents were used to controlelements within the BHS. Elements of interest were assignedlocal agents, diverters, and mergers, while single global agentswere given tasks, such as routing and communication. Divertagents would query the routing agent to determine which pathto send a bag along, while merge agents had the potential toprioritize the merging lines rather than using a FIFO rule. Thecentralized control scheme has not been completely replaced inthis situation as rather than a central controller now, there is acentralagentresponsibleforpathselection,asinglepointoffail-ure still exists, somewhat at odds with the work being presentedas decentralized. Rather local routing decisions are being madebased on information from a central source, much like the cur-rent operation of a BHS. The authors state that they were able toachieve more advanced utilization when compared to the src-inal central control strategy. These results are not quantified orthe srcinal control strategies are detailed. Problems with thecontrol strategy were described in some detail, the messagingprotocol between agents created too much overhead and a bot-tleneck formed around the messaging agent, seriously limitingthe ability for this method to expand to a larger BHS. Otherimportant details were mentioned, but no discussion about howthey can be managed, including early bags and the prioritizationat merges. These points, along with the idea of local routingdecisions are important points to highlight from the paper andrequire further investigation.This paper has expanded by Hallenborg [13]. Here, morefocus has been given to the interoperability of the agents inthe system, following the foundation for intelligent physicalagents (FIPA) guidelines for standardizing agents. The paperfocuses on the interaction amongst collaborative agents ratherthan how the agents collectively solve the problem of routingbags through the network. The routing control is acknowledgedto be centralized and the author is endeavoring to decentralize itinfuturework.Theroutingagentoperatesbycreatinganetwork graph representation of the system and maintaining informationon the traffic flows. Dijkstra’s shortest path algorithm is usedto determine path selection at each node much like a link-staterouting protocol. How traffic flows are used to modify weightsin the graph, used by the shortest path algorithm, is not exploredand is a central point in the technique used.III. S EARCHING FOR  P OLICIES The task of routing bags through a BHS is very easily likento that of routing data packets through a computer network. In acomputernetwork,sourcenodescreatedatapacketsandforwardthemviaanetworkofrouterstotheappropriatedestination.Theequivalent process in a BHS is to consider bags as data packets,diverters as routers, and the flow of data is from check ins toexit laterals.There are, however, contrasts that must be described. Firstly,data flow inanetwork can be, and ismostly,bidirectional. Here,we assume that all bag flow in the BHS is unidirectional, bagsflowfromsourcetodestinationonly.Avalidassumptionasbidi-rectional bag flow is a rare occurrence, and in instances, wherebidirectionalconveyors areused,asinindexingconveyors, usedinearlybagstorage(EBS),thebagflowcanbedescribedasone-way by adding a virtual node to a directed graph representationto ensure unidirectional flow.A second important difference between a computer network and a BHS is that intermediate points must be reached in theBHS prior to the bag reaching the destination. Additionally,these intermediate points are unknown when the bag enters thesystem. This requirement for bags to be directed along specificpathsinthesystemdiffersgreatlyfromthebasicnetworkroutingprotocols that are generally best-effort shortest path algorithms,i.e., open shortest path first (OPSF) and routing informationprotocol (RIP). The advent of policy-based routing has enabledthe ability to route-specific traffic along predefined paths in thecomputer network, exactly matching the need in the BHS toroute bags based on their status.Policy-based routing is the routing of traffic classes over spe-cificpathsthatobeypredefinedoperationalrequirements,whichmaybeconcernedwithperformanceorresource-utilization[15].In a computer network, a policy may be to route data belongingto video streams along a faster link than that used by email data.In a BHS, this idea applies to routing a bag toward a manualencode station after an ATR has failed to read the barcode at-tached to the bag. A policy can be concerned with performance,QoS, utilization, traffic engineering (TE), or both [15].Smith and Garcia-Luna-Aceves [15] argue that existing net-work implementations require mechanisms sitting above therouting layer to provide QoS and TE. This causes problemssuch as inefficient bandwidth allocation, since path creation isbased on shortest path routing and is unlikely to have any rela-tionship to QoS and TE demands. The authors develop a newclass of algorithm that supports QoS and TE inherently, whichoperates on a hop-by-hop basis. This hop-by-hop mode of op-eration is important when considering a BHS as the full set of resources, a bag must pass through is unknown when a bag en-ters the system, rather it is determined as the bag passes throughthe system. The hop-by-hop nature allows for routing decisionsto be made along the path as opposed to a source-based ap-proach that defines a path from source to destination that thedata packet must follow. Authorized licensed use limited to: DEAKIN UNIVERSITY LIBRARY. Downloaded on April 16,2010 at 06:11:46 UTC from IEEE Xplore. Restrictions apply.  192 IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS, VOL. 40, NO. 2, MARCH 2010 The algorithm policy-aware connectionless routing (PACR),developed by Smith and Garcia-Luna-Aceves performs threetasks: the computation and maintenance of routes that satisfythe QoS and TE constraints, traffic classification, and trafficforwarding. Of particular interest in this paper is that of routecomputation, specifically the creation of a set of routes from asource to a destination subject to network constraints.TheroutecomputationusedinPACRconsidersbothQoSandTE constraints. A link may only be used for a particular trafficclass if both QoS and TE parameters are satisfied.TE constraints are specified by expressions in Boolean alge-bra. A set of variables represent statements describing traffic orglobal state parameters that are either true or false. Smith andGarcia-Luna-Aceves used Backus–Naur form (BNF) grammarto represent these conditions. A specific administrative policyis described in BNF and applied to network links, restrictingtraffic, which is allowed to traverse this link. Note that the ad-ministrative policies for the entire network are known  a priori .The QoS constraints are based on link network performancemetrics, for example, delay and jitter. A traffic class has a per-formance measure set by an administrator and links, and perfor-mance below par, is not considered for routing purposes.To determine paths between source and destination, PACRoperates similarly to Dijkstra’s algorithm with an exception,rather than a single shortest path, multiple paths are found andonly paths that satisfy a policy, whether it will be QoS, TE, or acombination of both, are considered by the algorithm. What thealgorithm determines, is a set of routes to each destination foreach traffic class, where a path exists.This algorithm provides a foundation to create a new algo-rithmthatcan be used withina BHS. PACR assumes knowledgeof administrative policies imposed on the network. A twist tothis idea is that a modified algorithm operating in a BHS de-termines what policies are in place on the network and whatpolicy should be considered while making a routing decisionat a diverter. This approach is taken as it is, the resourcesplacedinlinealongconveyorsectionswithintheBHSthatdeter-mines the policy for that conveyor line, not something set by anadministrator.Our new algorithm BHS status-based routing (BSR), likePACRisbasedonDijkstra’salgorithm,wherePACRfindspathsto every destination for each traffic class and BSR determineswhat combination of resources are available from each diverteregress toward every exit. This information is used as a basis forroutingdecisionsatthatdiverter.Asabag’sstatusdeterminesitsprocessing requirements, this status drives the routing decision.After BSR has been run for the BHS, a diverter has knowledgeofwhatresourcesareabletodirectbagsandbasedonbagstatus,routing decision is made.The BHS is represented as a weighted oriented graph  G  =( N,L ) , where  N   is the set of nodes and  L  is the set of links. Let N  i be the successors of   i , where  i,N  i ∈ N  . A link   l ij  ∈ L  hascost  c ij . The BSR algorithm is shown in Fig. 1 and a notationguide is shown in Table I.Thealgorithmfinds k -shortestpathsforafinitesetofpoliciesfrom a given source node to all other nodes by traversing pathsin terms of cost in ascending order. Fig. 1. BSR algorithm.TABLE IBSR N OTATION  G UIDE The algorithm begins by adding the source node to a cost-based prioritized queue  T  . The algorithm takes the item withthe least cost from the queue and checks if a route already existsfrom the source to this node and adds the route to the set of permanent routes  P  , if not otherwise, the algorithm will check if the current policy has been found previously. If not or thecount of this policy is less than the  k  number of routes desiredto be found, then this new policy is added to  P  . Otherwise theneighbors of the current node are checked to see if they willcreate a new policy, if they are added to  T  .The policy referred to, in BSR is defined by the statuses abag can take. As previously described, a bag must be screenedand scanned before exiting. The resources within the network that provide these services, therefore, are required to set policywhen the BSR algorithm runs. The policy is defined as an array,each index representing a resource class. When a resource isencountered on a link, the corresponding array index is set to Authorized licensed use limited to: DEAKIN UNIVERSITY LIBRARY. Downloaded on April 16,2010 at 06:11:46 UTC from IEEE Xplore. Restrictions apply.  JOHNSTONE  et al. : STATUS-BASED ROUTING IN BAGGAGE HANDLING SYSTEMS: SEARCHING VERSES LEARNING 193 true. The sum of available policies is given by  2 P  , where  p  isthe count of resource classes.Where PACR finds possible loop free routes from a sourceto all destinations using administrative defined polices, BSRdiscovers what resources exist from a particular node to alldestinations. BSR also finds multiple paths for each policy.This difference exists as PACR is run online, as traffic con-ditions change, it will dynamically adjust routing strategies tosuit the TE aspects of the algorithm. BSR is designed to runoffline, providing routing control strategies for each diverter asthe knowledge assumed in PACR of known administrative con-straints, is unknown in the BHS. It is the algorithm responsibil-ity to determine which policies to be consider based on network architecture.IV. L EARNING  P OLICIES The previous section described a search method to determinewhat bag status was important while making a routing deci-sion. This section describes a method based on reinforcementlearning (RL) to similarly learn what bag status should be in-cluded while making routing decisions. The learning algorithmhas been dubbed BHS status-based learning (BSL).An RL agent, similar to the agent deployment by Hallenborg[13], controls each diverter in the system. The diverter agent isresponsible for maintaining a routing table and directing bagsaccordingtotheroutingtable.Asecondtypeofagent,aresourceagent, is used to provide feedback to the diverter agents. Theseagents control resources in the system that can change a bag’sstatus, i.e., an X-ray screening device.Stationary agents, as opposed to mobile agents, have beenchosen due to the nature of the problem. Mobile agents travelthrough the network, experiencing the delays, queues, and othernetwork nuances that influence performance. As the network isexperienced, the mobile agents update routing tables with theknowledge they have gained. The agents may be simple ant likeagents [16], or more complex entities [17]. In a BHS environ-ment, the function of a routing packet is not applicable. Mobileagents move through a computer network by making use of links between network nodes, they act just like a data packet. If the controlling system of a BHS was suddenly to start injectingphantom bags to collect network information, system perfor-mance would degrade, therefore, the ability to collect network state information must be achieved through means other thaninjected packets or mobile agents. Stationary agents are ableto operate in the BHS through direct and indirect communica-tion. As in Antnet [16], ants keep a record of nodes that theyhave visited, bags can be tagged with nodes that they visit, pro-viding indirect communication for information gathering. Di-rect communication with agents in a BHS has been extensivelystudied [13], and a similar mechanism can be implemented toprovide feedback between agents. Thus, the task of commu-nication between agents is summarized as diverter agents tagbags with their own identification, while resource agents querythis information place on bags and directly communicate withdiverter agents to provide feedback on route selection. Fig. 2. BHS schematic showing a divert toward a manual encode station. Theagentsgoalistodeveloproutingtablesateachdivertpointthat will successfully route a bag through the network based onbag status. A successfully routed bag will not required traverseresources determined by the bag’s status. Additionally, a bagmust arrive at the correct destination. The diverter agents learna routing policy through feedback provided by resource agents.Like the BSR algorithm presented in the previous section, thelearningagentswillprovidearoutingpolicybasedonbagstatus,rather than searching through the network, RL techniques willbe used and the different solutions will be analyzed.The feedback provided by the resource agents has been in-spired by the trail laying action of ants. Pheromones have beenusedtocreateshortestpathroutingalgorithms[16],butthemorerecent discovery of negative pheromones [18] can be used morequickly to discover a set of rules to base routing decisions on.The basis of the feedback signal is to send a negative signalwhen a bag arrives at a resource and the bag’s status indicatesthat it should not have been directed toward this resource. InFig. 2, the manual encode station ME is a resource agent, pro-viding feedback to upstream diverters. The diverter can routebags toward the ME or it can skip this process and route bagsdirectly toward the merge. In this simple example, it is obviousthat the ME resource agent will only provide feedback to thediverter when a bag is routed toward it whose status does notrequire manual encoding. Like the negative pheromone actingas a no entry signal, the negative feedback will prevent bagsof like status being routed along inappropriate paths. Resourceagents provide feedback on what status they themselves act ona bag, for example, a level 1 screening resource will providefeedback if a bag reaches it that does not require level 1 screen-ing. Additionally, resource agent still operate on bags, althoughat different rates. A level 1 screening resource may normallypass 80% of bags, in order to explore the network with bags of varying status, all possible outcomes of a resources action ona bag have equal probability, i.e., for the level 1 screener, 50%bags pass.The diverter agent is run in a purely exploratory manner. ThetradeoffinRLwithexploitationisnotafactor,asthealgorithmisused to learn valid routes through the negative feedback system.Positive feedback in communication networks has been usedto develop adaptive routing algorithms [19], [20], indeed, thisalgorithm could be modified to include positive feedback inorder to route along shortest paths, but the goal of the algorithmistoidentifywhatstatusadivertershouldconsiderwhilemakinga routing decision.The routing decision is randomly chosen over availableroutes. All egress points from the diverter are considered validuntil feedback is received. The granularity of the routing tablemust consider the entire range of bag status and all destinations Authorized licensed use limited to: DEAKIN UNIVERSITY LIBRARY. Downloaded on April 16,2010 at 06:11:46 UTC from IEEE Xplore. Restrictions apply.
Recommended
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks
SAVE OUR EARTH

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!

x