A proactive decision support method based on deep reinforcement learning and state partition

摘要

Big streaming data is an important kind of big data which we need new technology to process. Getting knowledge from online streaming data and making decision online can help us get more value from Big data. A proactive decision support system can predict future states and mitigate or eliminate undesired future states by taking some actions proactively. But it is difficult to handle some issues like the data distribution change in streaming data, combination of prediction and decision making, and the huge state space in decision making. In this paper, we propose a proactive decision support method based on deep reinforcement learning and state partition. The predictive analytics part uses deep belief networks with two level incremental training method. The deep reinforcement learning part uses deep belief networks as function approximation which is learned by semi-gradient method. Off-policy is supported through important sampling. Two kinds of state partition and parallel execution methods are proposed to improve the performance. The experimental evaluation in traffic congestion control application shows this method works well in both accuracy and performance.