Isamun

arXiv:2408.05216v1 Announce Type: new Abstract: Air pollutant exposure kills over 6,700,000 people it per annum, yet there remains a systemic lack of accurate ground level data reporting the concentrations of the leading causes of such fatalities. Ambient particulate matter is a primary driver of this effect. Namely, PM1.0, PM2.5, and PM10.0 display a systemic lack of accurate and high-definition reporting. This project suggests and implements a prototype for a distributed and low cost model for reporting such data and designs a novel framework in order to remedy three main shortfalls of previously implemented systems. First, their central operation and distribution, and therefore their requirement of trust in a central governing body. Second, their requirement of the purchase of comparatively high-cost devices for ordinary consumers. Finally, their high degree of error and accordingly low functional certainty. This project explores the creation of AirChain, a prototype system utilizing blockchain technology that will demonstrate the effectiveness of low-cost sensors when paired with simple microcontroller devices.

Deep Q-Networks algorithm (DQN) was the first reinforcement learning algorithm using deep neural network to successfully surpass human level performance in a number of Atari learning environments. However, divergent and unstable behaviour have been long standing issues in DQNs. The unstable behaviour is often characterised by overestimation in the $Q$-values, commonly referred to as the overestimation bias. To address the overestimation bias and the divergent behaviour, a number of heuristic extensions have been proposed. Notably, multi-step updates have been shown to drastically reduce unstable behaviour while improving agent's training performance. However, agents are often highly sensitive to the selection of the multi-step update horizon ($n$), and our empirical experiments show that a poorly chosen static value for $n$ can in many cases lead to worse performance than single-step DQN. Inspired by the success of $n$-step DQN and the effects that multi-step updates have on overestimation bias, this paper proposes a new algorithm that we call `Elastic Step DQN' (ES-DQN). It dynamically varies the step size horizon in multi-step updates based on the similarity of states visited. Our empirical evaluation shows that ES-DQN out-performs $n$-step with fixed $n$ updates, Double DQN and Average DQN in several OpenAI Gym environments while at the same time alleviating the overestimation bias.

今日の午後には、開会式です。 その前にサイト改善とゲート作り。 全員、しっかり働いています。