Discovering urban mobility structure: a spatio-temporal representational learning approach

ABSTRACT The urban mobility structure is a summary of individual movement patterns and the interaction between persons and the urban environment, which is extremely important for urban management and public transportation route planning. The majority of current research on urban mobility structure discovery utilizes the urban environment as a static network to detect the relationship between people groups and urban areas, ignoring the vital problem of how individuals affect urban mobility structure dynamically. In this paper, we propose a spatio-temporal representational learning method based on reinforcement learning for discovering urban mobility structures, in which the model can effectively consider the interaction knowledge graph of individuals with stations while accounting for the spatio-temporal heterogeneity of individual travel. The experimental results demonstrate the advantages of individual travel-based urban mobility structure discovery research in describing the interaction between individuals and urban areas, which can account for the intrinsic influence more thoroughly.


Introduction
Humans play a pivotal role as the central component of urban transportation systems.With the ongoing expansion of the transportation sector, the influence of human travel activities on the configuration and structure of urban areas has intensified.This transformation is evident in the shift from traditional urban monocentric patterns to the emergence of urban polycentric arrangements and concentrated hotspot zones (Anas, Arnott, and Small 1998;Gordon, Richardson, and Wong 1986;Greene 1980;McMillen and McDonald 1997;Pan et al. 2018;Wang et al. 2020).Cities, functioning as intricate network systems, are characterized by a multitude of dynamic population movements.These movements contribute to the creation of diverse patterns of human mobility within metropolitan environments, exhibiting variations across different geographical locations and temporal periods.Goodchild and Janelle (1984) has defined cities as sophisticated network systems, capturing the intricate interplay of elements within urban settings.Investigating the intricacies of urban structure and the evolving patterns shaped by human transportation activities has become a central focus of contemporary research across disciplines such as urban geography, computer science, and demography.The exploration of urban mobility structures serves to unveil the underlying dynamics of human activities within cities, analyze the travel behavior exhibited by urban residents, and elucidate the nuanced impacts of urban spaces from a microscopic vantage point (Große et al. 2018;Hasanzadeh, Kyttä, and Brown 2019;Jiang, Yin, and Zhao 2009).The pursuit of understanding urban mobility structures holds significance not only on a localized scale but also from a broader perspective.Such insights are crucial for comprehending urban growth trajectories, devising effective urban traffic management strategies, optimizing bus route planning, and providing a comprehensive overview of economic mobility patterns.
Urban areas have emerged as bustling centers of human engagement, and the availability of accessible transportation alternatives has significantly enhanced mobility, broadening the spectrum of activities accessible to local residents.Consequently, a wealth of data pertaining to human movement is now being collected, encompassing communication records, tracking data (including GPS data from taxis and public transit), and positional data gleaned from cell phones (such as locationbased updates on platforms like WeChat and Weibo).A notable illustration of this data abundance is evident in census records, revealing that the average daily ridership of public transportation systems in major cities like Beijing and Shenzhen has surged beyond the five million mark.These records not only furnish insights into urban whereabouts and temporal trends but also harbor potential to unveil intricate facets of residents' travel patterns, encompassing preferences and even emotional states (Shelat et al. 2023).
Furthermore, human activities wield considerable influence over urban structures, leading to modifications and alterations.Consequently, we characterize alterations in urban layout and network attributes resulting from human actions as the 'urban mobility structure.'Scholars across various disciplines have embarked on a comprehensive exploration of this phenomenon, adopting distinct perspectives: (1) Statistics: Researchers engage in identifying and analyzing regions by quantifying the flow of economic and commercial activities amid different locales (Cebollada 2009;Liu et al. 2015;Wei et al. 2020;Yajing et al. 2016).(2) Complex Networks: An approach involving the division of cities into distinct areas through techniques like grid-based segmentation and road networks.This technique entails investigating relationships among regional networks, treating regions as nodes, population movements as edges, and gauging the strength of connections as weights.Noteworthy methodologies encompass Modularity (Newman 2006), Louvain (Radicchi et al. 2004), InfoMap (Radicchi et al. 2004), among others (Rosvall and Bergstrom 2007).(3) Data-Driven Machine Learning Techniques: These methods hinge on data as the foundation for autonomously uncovering concealed insights within mobility data.Examples of such approaches include deep clustering (Li et al. 2021;Murakami and Yamagata 2017), aggregation techniques (Guo 2008;Yuan et al. 2014), and graph embedding (Wang et al. 2015;Yao et al. 2016;Yuan et al. 2014).Despite the advancements, certain limitations persist within these studies.For instance, the heterogeneity of individual travel behaviors and interactions with urban spaces often remains inadequately addressed.This shortfall impedes these methods from unearthing deeper layers of insight from the amassed data.
A fresh approach to discerning metropolitan mobility structures and patterns can be achieved by devising a spatio-temporal representation model grounded in individual travel dynamics.This approach holds the potential to address two pivotal aspects.Firstly, it allows for the integration of the intricate spatio-temporal variations characterizing people's movements into a compact low-dimensional space.In this schema, the vector within this low-dimensional space serves as the repository for the initial heterogeneous data pertaining to individual travel behaviors.This strategy effectively captures and encapsulates the multifaceted nature of individual mobility.Conversely, the latent space thus established facilitates the mapping of interactional data between individuals and urban locales during transit.By leveraging spatio-temporal representational learning, this model enables the nuanced and flexible portrayal of diverse urban regions.
This innovative approach not only adeptly navigates the intricate realm of human mobility but also presents a holistic avenue for conveying the nuanced attributes of urban landscapes.This, in turn, enriches our grasp of metropolitan dynamics.The present paper introduces a novel method for representational learning rooted in reinforcement learning and guided by the perspective of individual travel.This methodology explicitly incorporates the spatio-temporal variations inherent in individual travel, aiming to synergistically map the dynamic and interactive facets onto the representation space.The research makes three significant contributions: (1) Integration of individual knowledge graphs: The study generates individual travel knowledge graphs that establish connections between individual, stations and urban areas.These graphs serve as foundational insights, directing the reinforcement learning process to acquire representations for both individuals and urban areas.In sum, this study not only introduces a pioneering approach but also provides a structured framework for understanding urban dynamics by holistically capturing the intricate interplay between individual mobility and urban characteristics.

Representational learning
Recently, representational learning has emerged as a prominent field of study in computer science research.The primary objective is to embed the graph G n×n into a low-dimensional space R n×d , where d<<n, and the resulting structure retains the information from the original graph.This transformation aims to retain the intrinsic information of the original graph, providing a foundation for various downstream tasks including clustering, classification, edge prediction, and recommendation.
This field exhibits progress through three broad categories of methods: factor decompositionbased methods, random walks-based methods, and deep learning-based methods (Goyal and Ferrara 2018).To preserve node similarity, the graph is translated into matrices through techniques such as component decomposition, involving domain matrices, Laplace matrices, as well as techniques like Locally Linear Embedding (LLE) (Roweis and Saul 2000) and High-Order Proximity preserved Embedding (HOPE) (Ou et al. 2016).
Random walk-based approaches, such as DeepWalk (Perozzi, Al-Rfou, and Skiena 2014) and node2vec (Grover and Leskovec 2016), revolve around traversing the network in a continuous and iterative manner, forming complete paths.This process implicitly maintains node similarity while capturing local context information from the graph.Concurrently, as the domain of deep learning expands, numerous methodologies leveraging deep neural networks have been applied to graph representation.The depth auto-encoder, which can model the nonlinear nature of data, finds utility in techniques like Deep Neural Graph Representation (DNGR) (Cao, Lu, and Xu 2016), Structural Deep Network Embedding (SDNE) (Wang, Cui, and Zhu 2016), Variational Graph Autoencoders (VGAE), and Graph Autoencoders (GAE) (Do, Nguyen, and Deligiannis 2020).
By introducing neural networks within the graph structure, these methods can learn node similarity with a high degree of generalizability.This addresses challenges like the effective computation of sparse graphs.As deep learning continues to evolve, these techniques contribute significantly to enhancing graph representation and its applicability to diverse scenarios.

Urban mobility structure
Urban structure encapsulates the spatial relationships and distribution patterns of geographical elements within a specific territorial region.This intricate configuration evolves as a cumulative outcome of human spatial behaviors and location preferences over time (Rossi-Hansberg and Wright 2007).Conversely, the urban mobility structure delves into the repercussions of human activities on urban spaces.This entails examining the influence of human travel patterns and regulations on urban areas, alongside a comprehensive exploration of the interactions between individuals and urban environments (Jiang, Ferreira, and Gonzalez 2012).For instance, this encompasses dissecting typical mobility structures, such as the commuting-to-work-home pattern in residential zones, and the mobility arrangements of commercial areas catering to leisure and entertainment activities.Nonetheless, the diversity in people's spatial and temporal travel requisites in urban locales introduces complexities that underpin the transformation of urban mobility structures.These complexities manifest differently across distinct time periods, reflecting the dynamic nature of urban movement patterns.
Initially, many scholars perceived urban structure as static and consequently directed their attention towards exploring how this structure influences individual travel behavior.This involved studying alterations in individual travel behaviors through the amalgamation of land use and settlement dimensions (Rickwood et al. 2008), as well as evaluating urban policies by analyzing individual travel behaviors (Cochrane and Ward 2012;Hillman 1975).However, the perspective began to shift in the 1970s, as researchers recognized that cities are dynamic systems characterized by intricate exchanges of material, energy, and information both within and between metropolitan areas.Moreover, the interconnectedness of urban spaces is exemplified by individual travel activities, which weave together discrete physical resources into a cohesive network.Consequently, the influence of individual travel behaviors on urban structure garnered attention from geographers (Shibayama 2011;Yue et al. 2009), prompting the exploration of cities beyond the spatial distribution of physical environments and economic resources.In essence, geographers began advocating for a broader understanding of cities that encompasses population mobility patterns, such as commuting patterns (Modarres 2011;Yao et al. 2021), isolation patterns within activity spaces (Zhang et al. 2021a), and travel models in the context of epidemics (Gao et al. 2020;Shanthappa, Mulangi, and Manjunath 2023), all of which contribute to shaping the urban activity structure.Nonetheless, advancements in this research domain have encountered constraints related to data sources, analytical methodologies, and computational capabilities.Consequently, while studies pertaining to the organization of urban mobility have mostly concentrated on urban morphology (Zhong et al. 2014), there remains untapped potential for more comprehensive investigations due to these limitations.
As urban areas continue to expand in size and computational capabilities advance, numerous geographers are harnessing large-scale individual travel data to explore various aspects of urban dynamics, including urban centrality, functional zoning, and interactions between residents and the city.For instance, Shibayama (2011) leveraged smart card data to uncover a polycentric urban structure, unveiling intricate and multifaceted connections between different cities.Jiang, Ferreira, and Gonzalez (2012) employed statistical techniques to identify spatio-temporal clustering effects within the intra-city population of Chicago.They proposed an extension of the perception of urban structure beyond spatial dimensions, incorporating the temporal dimension.Yuan et al. (2014) employed a data-driven methodology that divided the city into distinct units using road networks.They then fused Point of Interest (POI) data and taxi trajectory data to discern the functional attributes of each unit within the city.Zhong et al. (2014) adopted a network science framework to analyze smart data collected from Singapore over several years, uncovering correlations between the development of urban centers and population mobility.Nonetheless, existing research often lacks a comprehensive perspective from the standpoint of individual travel.This limitation poses challenges in achieving a complete and precise understanding of individual mobility patterns and their interplay with urban structure.To address this gap, a more integrated approach that places individual travel behavior at the core is necessary to attain a comprehensive depiction of urban dynamics.

Individual mobility representation
Individual mobility representation refers to the procedure of projecting individual travel data onto a lower-dimensional space for each individual.This transformation is performed in a way that ensures the vectors within this reduced-dimensional space encapsulate the essential details of the original individual travel information (Fu and Lee 2020).The outcomes of individual characterization, as derived from this process, can be effectively employed to fulfill various tasks.Examples include urban population segmentation (Zhang et al. 2019), individual travel prediction (Wang et al. 2020), and anomaly detection (Chen et al. 2014;Sillito and Fisher 2008).
Current research in individual representation is still in its nascent stages, primarily focused on the spatiotemporal representation and modeling of individual trip trajectories.The core objective is to transform trajectories of varying lengths into vectors of consistent lengths.These vectors encapsulate key details such as topology, geographical coordinates, environmental context, and other pertinent information from the original trajectories.For instance, Gao et al. (2022) have introduced GraphTUL, a semi-supervised model that incorporates an adversarial network with policy gradient techniques to enhance identification capabilities.This model effectively utilizes both labeled and unlabeled trajectories to address the challenge of insufficient labels.Fu and Lee (2020) have devised a concept called road-trajectory cross-scale contrast, which functions to establish connections between two distinct scales by maximizing overall mutual information.Additionally, Ghosh, Ghosh, and Buyya (2020) have proposed a framework known as MAR (Mao et al. 2022), presenting a distinctive representation learning method that focuses on learning trajectory embeddings within the context of road networks.The crux of individual representation is heavily reliant on trajectory data, which serves as a critical and fundamental data source derived from individual travel activities.As research in this domain advances, it holds potential to significantly contribute to our comprehension of individual mobility patterns and their broader implications within urban contexts.
Indeed, individual travel generates a multifaceted web of interaction information that encompasses factors such as journey duration, spatial dynamics, individual variations, and activity contexts, supplementing the trajectory data.This expanded scope of information necessitates elevated standards for the investigation of individual travel representation.The challenge lies in how to harness this information in a judicious and effective manner.Efficiently incorporating this comprehensive set of data entails responsible and thoughtful consideration.Researchers must navigate the intricacies of managing and processing this complex interaction information, while also ensuring that the resulting representation accurately captures the rich and diverse aspects of individual mobility patterns.This calls for the development of advanced methodologies that can successfully handle the multifarious dimensions of individual travel data, ultimately contributing to a more comprehensive understanding of urban dynamics and individual behaviors.

Reinforcement learning in urban research
Reinforcement learning, distinct from traditional supervised and unsupervised learning approaches, embraces a 'trial and error' methodology, mirroring how agents learn through interaction.This approach aligns more closely with the cognitive processes humans employ to understand their surroundings (Ghavamzadeh et al. 2015).In essence, reinforcement learning involves agents taking actions within an environment and learning from the outcomes of those actions.Throughout this process, agents engage in actions by interacting with their environment and are subsequently rewarded based on their choices.A higher reward value signifies more favorable and accurate agent actions.The essence of reinforcement learning lies in the interplay between agents and their environment, a relationship that can be conceptualized by the structure (S, A, P, R, r).Here, S represents a finite set of states, A signifies a finite set of actions, P denotes a state-transition matrix, R encapsulates a reward function, and r stands for a discount factor governing cumulative rewards.This holistic structure orchestrates the learning process, allowing agents to navigate the environment and optimize their actions based on received rewards.
Currently, the fusion of reinforcement learning with deep learning has given rise to a potent predictive model that finds application not only in gaming scenarios (Pang et al. 2019;Silver et al. 2016) and robot control (Kober, Bagnell, and Peters 2013;Ng 2011) but also plays a pivotal role in processing spatio-temporal data (Li, et al. 2019;Wei et al. 2018).In this realm, Shang et al. (2022) have introduced a low-cost deep graph reinforcement learning network aimed at predicting spatiotemporal shifts in traffic flow.Zhao et al. (2023) leveraged reinforcement learning to allocate space-time tasks, showcasing its versatility in dynamic resource management.Moreover, Fan, Jiang, and Mostafavi (2021) harnessed deep learning to assess disruptions in urban mobility patterns during crisis scenarios.These instances collectively underscore the potential of reinforcement learning in tackling intricate spatio-temporal challenges.As the synergy between reinforcement learning and deep learning continues to evolve, it holds the promise of addressing multifaceted spatial and temporal complexities across a diverse array of domains, opening avenues for enhanced understanding and solutions in various fields.
Certainly, the intricate dynamics of population mobility within urban environments, coupled with the intricate interplay between mobility and various activity scenes, present a complex challenge.These complexities underscore the pressing need to enhance reinforcement learning methodologies to better align with the requirements of current problems in this domain.Improving reinforcement learning techniques is imperative to address these challenges effectively.This may involve devising novel approaches that can accommodate the multifaceted nature of urban mobility patterns, considering the interdependencies between various urban activities, and ensuring that the learned models are capable of robustly capturing the intricacies of these interactions.By tailoring and advancing reinforcement learning to cater to these complexities, researchers can forge more accurate, comprehensive, and adaptable models that can better tackle the unique demands of analyzing population mobility within urban spaces.

Spatio-temporal representational learning based on reinforcement learning
In our study, we establish two essential knowledge graphs: the intersection knowledge graph encompassing individuals and stations, and the affiliation knowledge graph linking stations with city areas.These knowledge graphs are constructed based on actual individual travel behaviors.We deploy reinforcement learning techniques to process these knowledge graphs, with the overarching objective of dynamically updating representations for individuals, stations, and city areas.This evolution involves transitioning from separate, independent representations to a cohesive framework of dynamic updates.
A distinguishing facet of our research is the incorporation of spatio-temporal heterogeneity inherent in individual travel patterns.To navigate this intricate landscape, we introduce the concept of generalized anisotropy (Berkowitz and Takano 1999;Deng et al. 2019) as a guiding principle for our reinforcement learning model.This concept informs and shapes the way the reinforcement learning model updates the representations of individuals, stations, and city areas.By doing so, we aim to create a comprehensive approach that captures the intricate dynamics of individual travel behavior and their interactions with urban spaces, paving the way for a more refined and accurate understanding of urban mobility patterns.

Key elements in reinforcement learning
Agent can imitate the behavior of an individual and provide personalized action prediction for an agent based on the current environment and state.
Action is the intersection event of agent, i.e. an individual visits station.Formally, a i,j means that the action of agent i visit point j.Then, when agent visits p j at moment l, it can be expressed as Environment is defined as a dynamic representation rule based on individual travel knowledge graph, on the one hand, the visiting behavior of individuals affects the representation of stations in the city, on the other hand, the stations in the city also affect the travel representation of individuals.According to the access relationship between individuals and scenes and the affiliation relationship between scenes and urban areas in the knowledge graph, the dynamic representation rules of individuals and scenes and scenes and urban areas can be formulated comprehensively and accurately.
Reward: the accuracy of an intelligent body visiting a station can be divided into the proximity of the location of the visited station and the consistency of the ID of the station, then the reward obtained by the action of the agent visiting the station can be divided into (1) r d , the inverse distance weighted sum, i.e. the true location of the visited station inverse of the distance from the predicted location, and (2) r p , the difference between the predicted ID and the true ID of the visited station.Then, the reward can be expressed as, Where l d and l p are the weight of r d and r p respectively.

Joint individual-station -urban area spatio-temporal representational learning algorithm
In this paper, we consider the interaction between individuals and stations during the individual travel process, and perform joint dynamic representation of individuals and stations with the help of a reinforcement learning model.We must integrate the access relationship between individuals and stations in the dynamic characterization process, and they serve as anchor points for each other.And we take into account the heterogeneity of the individual travel process in the dynamic representational learning process, and introduce the concept of anisotropy to guide the process of individual travel dynamic representational learning, which provides the results of individual and station representation with spatio-temporal heterogeneity information to achieve the comprehensiveness and objectivity of individual-stations dynamic model, such as Figure 1.
In this research, we use the TransD model to initialize the representation of individuals and stations based on the knowledge graph of the interaction between individuals and stations.The dynamic joint representation of individuals, stations, and urban areas is then conducted using reinforcement learning.

Policy design of spatio-temporal representational learning
Deep Q networks are widely used for strategy learning in reinforcement learning by using deep neural networks.In general, traditional deep Q networks can only handle vectors or matrices, and it is difficult to use knowledge graphs as inputs.Therefore, this paper designs a hierarchical pooling method to combine individual travel representations and scene representations as the loss of deep Q networks, as shown in Figure 2.
We process the individual-station intersection knowledge graphs in each time period into vectors, but the current graph pooling method is not suitable for individual-station heterogeneous relationships.Therefore, the average pooling of the two knowledge graphs is used to generate vectorized representations of the heterogeneous nodes, i.e. individual travel representations, and station representations, respectively.Finally, in each time period, the average pooling is used on the above two vectors and connected to form a new vector representation, h con = concatenate(h u , h p ), h u , h p are individual and station representations respectively, and concatenate( • ) is the connection method.Then the connected h con is input into the fully connected network, which maps the given state (s) to Q(s, a) in one of the set of relations in the knowledge graph, and the strategy selects the scene with the highest Q(s, a) as the prediction result.
Intuitively, the higher the reward of the station visit action a l , the better the next visit to the intelligent body imitating the individual, then the greater the contribution of the data sample to the strategy training.Therefore, for each data sample (s l , a l , r l , s l+1 ), the reward-based priority score prio r is defined as the reward r l , prio r (s l , a l , r l , s l+1 ) = r l (2) In addition, the time difference (TD) is initially set for updating the DQN, and the larger the time difference (TD), the more valuable and informative the data sample is for the next intelligence learning.Therefore, we define the time error as the priority score, Where g is the balance coefficient.

Spatio-temporal representational learning
The process of individual-station -urban area dynamic representation includes two aspects, on the one hand, the individual are used as anchor points to update the representation of station, and on the other hand, the visited station is used as anchor points to complete the dynamic representation of individual trips and urban area.
(1) Anisotropy-based representation of individual dynamics Individual travel faces the challenge of heterogeneity in direction and rate changes, i.e. anisotropy in travel among people.We introduce each heterogeneous diffusion model in this research to explain each individual's travel circumstance, and then utilize the anisotropic diffusion model as a guide to map the individual travel heterogeneity to the representational results.The anisotropic diffusion model expresses heterogeneity in both direction and rate, and the equation is as follows: I t is the state at the previous moment, I t+1 is the state at the next moment, and the dispersion equation is the bias derivative in four directions with λ as the weight to regulate the diffusion intensity.The partial derivatives in the four directions are that, where I x,y is the state of the current position, x, y denote longitude and dimension respectively.cN x,y , cS x,y , cE x,y , and cW x,y are the coefficients in the four directions of north, south, east and west respectively.
where k is the weight, which is used to adjust the strength of the travel.
In practice, the attractiveness of the two places influences the rate of individual travel (each anisotropic diffusion), and the more appealing the urban area is to the individual, the higher the individual's inclination to travel.Then, in the anisotropic diffusion model, we can replace with the two-location attractiveness indicator.As a result, we update the individual representation model using the anisotropy concept.
Where, W u is the weight of the interaction between individual i and station j used to update the individual travel representation, h l ui and h l pj are the representations of individual and station at moment l, respectively, Dir is the four directions of southeast-northwest, and c p is the attraction between individual i at moment l and two places at moment l + 1 in the direction of p. Since the gravitational model can express the attraction situation between two places, we calculate the attraction situation between two places by improving the traditional gravitational model, Where, G poi,d denotes the number of destination stations, the higher the number indicates the better the economic condition of the destination, L b , L s denotes the total number of bus lines and subway lines between origin and destination places, t od,b and t od,s are the time spent from origin to destination by bus and subway respectively, w 1 and w 2 denote the proportion of population taking bus and subway.We take the result of the last moment individual dynamic representation as the spatio-temporal representation of individual travel, and write H u,f = [h l+1 u,1 , h l+1 u,2 , . . ., h l+1 u,n ], and n is the number of individuals.
(2) Dynamic station representation The dynamic representation process of station is completed on the basis of individual travel representation.The entire process is built on an individual travel knowledge graph, in which individuals visit station, tap the interaction relationship between individuals and stations, and realize the dynamic update of station as a result of this relationship.Then, based on individual travel representation, the station update formula is that.
This process occurs at the moment when j-th individual visits i-th station.h l p,i is the representation of the i-th station at moment l, and W p is the weight of updating the station with the individual as the anchor point.We take the result of the dynamic representation of the last moment station as the spatio-temporal representation of the station, denoted as H p,f = [h l+1 p,1 , h l+1 p,2 , . . ., h l+1 p,m ], and m denotes the number of stations.
(3) Dynamic urban area representation The affiliation knowledge graph of station and urban area connects two heterogeneous entities, station and urban area, and establishes the knowledge graph between the two through the affiliation relationship.After that, there should be a one-to-one correlation between station and urban area from the entity space mapping to the representation space, i.e. h z = h p + h b .As a result, the representation of the affiliation relationship between station and urban area is established in the updating process of urban area.
where h l+1 z,i is the representation of the i-th urban area at instant l + 1, h l+1 p,i is the representation of the i-th station located in the urban area, and h b is the outcome of the affiliation relationship.As the spatio-temporal representation of the urban area, we use the outcome of the dynamic representation of the last moment area, denoted as and t is the number of urban areas.

Robust continuous clustering-based urban mobility structure detection
The outcomes of the representational learning in Section 3.3 form the basis for the discovery of the urban mobility structure.Urban area are given uniform distribution features and consistent size via representational learning.As a result, clustering and other techniques can be used to mine the structure.
By using Euclidean distance, traditional clustering finds nodes with comparable properties.Since all conventional clustering algorithms work with two-dimensional point data, the closer two points are to one another, the more similar they are, and the more likely it is that they would cluster together into the same class.Urban mobility structure discovery study must deal with multidimensional data, and the output of conventional multidimensional clustering algorithms is less comprehensible and necessitates the input of several parameter values, such as the number of clusters for K-means clustering (MacQueen 1967) and the minimum distance for DBSCAN clustering (Ester et al. 1996).
The Robust continuous clustering (RCC) algorithm (Shah and Koltun 2017) can find the number of clusters adaptively and does not rely on prior information of the true number of clusters.A global continuous objective optimization process built on robust estimation can be used to describe robust continuous clustering, a straightforward, quick, and effective clustering algorithm.Since the fundamental function of the robust continuous clustering approach used in this paper to identify the urban mobility structure discovery can be described as, where z i is the representational result of the i-city, H z,f = [h z,1 , h z,2 , . . ., h z,n ] is the set of city representations, n is the number of cities, and Y = [y 1 , y 2 , . . ., y n ] is the learned type from H z,f .The clustering is obtained by optimizing Y. 1 denotes characterizes the city pairs with high similarity using the m-KNN (Brito et al. 1997) calculation.The weights v i j balance the contribution of each data point pair, expressed as the normalization of urban GDP.l is the weight between different terms, and l RCC,ij is a regular term to control the generalization ability of the model.
Where, C = (i,j)[1 v i j l RCC,ij (e i − e j )(e i − e j ) T .
Equation 9 can be easily expanded to datasets with tens of thousands of dimensional samples and can produce effective and scalable optimization results using least-squares iteration.While Y is fixed, the best solution can be achieved by decoupling each l RCC,ij ; when L is fixed, it is converted into a least squares issue.Accordingly, the optimization of robust continuous clustering can be seen as the process of optimizing (Y, L) individually.This structure calls for the alternative updating of Y and L to achieve optimization, and Figure 3 depicts the resilient continuous clustering process.
We implemented the Reinforcement Learning in PyTorch 1.9.0 by Python 3.8 and applied gradient descent with Adam optimizer for Deep Q-Network training.The outputted 128-dimension embedding vectors for urban areas are used as the inputs for the clustering algorithm.We tune the weighting parameters l d = 1e − 5, l d = 0.6 in Eq. ( 1) to produce the best results for different experimental settings.The validation dataset was also used to fine tune the learning rate and 0.001 was recommended.We leveraged robust continuous clustering to extract mobility structure.

Data description
Shenzhen, a pioneer of China's reform and opening-up, has a population of more than 12.5 million and covers an area of more than 2,000 square kilometers, adjacent to the Hong Kong Special Administrative Region.Shenzhen has the most complete bus and subway system in China, including 8 main subway lines with a total of 199 subway stations and 808 bus lines with 6,226 bus stops, as shown in Figure 4.
In this research, bus trips are reconstructed using SCD, bus trajectory data, bus network, and road information by developing temporal and spatial rules for trip chains (Zhang et al. 2020).Table 1 shows the journey time, travel location, arrival time, arrival location, and transit stops during the period of April 3 to April 9, 2017.
Throughout the week, over 40 million records were collected, of which the POI data is the layer data of 2017 Gao De Map.The specific data descriptions are shown in Table 2.
Shenzhen is divided into nine administrative districts and one functional district.The center districts of Shenzhen are typically considered to be Luohu, Futian, and Nanshan, with a trend of expansion to Baoan, Longgang, and Longhua.The core district is divided among dense business and residential neighborhoods, among other things.Residents can drive short distances to their workplaces and leisure facilities due to the different land uses in the city core.Nonetheless, most suburban residents commute great distances via the metro system during the week, owing to the abundance of job prospects in the center portions of Luohu, Futian, and Nanshan districts, as well as the southern and western parts of Longhua and Longgang districts.Industrial areas and urban villages remain the dominating land use patterns in several suburban and exurban areas (for example, Baoan, Guangming, Longhua, Pingshan, and the northern portion of Longgang District).These locations are characterized by a concentration of temporary workers and urban villagers who utilize public transportation less frequently than residents of downtown areas.
The POI distribution has obvious spatial heterogeneity.Shenzhen has 54,897 commercial points and 194 public facilities, the majority of which are located in the urban area (Futian, Nanshan, and Luohu districts); some small commercial points are scattered throughout the residential areas of Baoan, Guangming, Longhua, and Longgang districts; educational points (3540), government institutions (5394), and medical services (7520) are also distributed unevenly: Other districts have and medical possibilities are few, and in Pingshan and Dapeng, there are few commercial and educational facilities, with clusters of tourist attractions (186) in Bao'an, western Guangming, Longhua, central Pingshan, northern Longgang, and distant parts of Dapeng.We first gridded the city of Shenzhen with 100 m × 100 m grids to develop the urban areas.Cells with similar traffic patterns are merged to decrease total computation and provide maps with continuous accessibility.The specific implementation procedure consists of the following steps: (1) identifying each grid as a boarding point and establishing the maximum walking distance for bus and subway stops as 400 and 1000 meters, respectively, as the threshold value using Dijkstra to determine the walking distance from the grid's center of mass to surrounding stations.(2) creating a vector of adjacent stations for each grid cell, i.e. recording the IDs of all stations within walking distance as the vector of each grid cell.(3) After calculating the similarity of the nearby grids (eight neighborhoods), the grids with high similarity are merged and given new IDs, resulting in 18,108 urban areas (Zhang et al. 2021b).There is a situation where a station serves multiple areas, but it does not affect the updating of the representation of the site city area.

Urban mobility structure on weekdays and weekends
Urban mobility structure discovery was carried out for weekdays and weekends in Shenzhen based on the representational learning results of urban areas, and the detection results were assessed individually.
The urban areas on weekdays (Monday to Friday) and weekends (Saturday and Sunday) are adaptively divided into five groups based on robust continuous clustering algorithm (RCC).Individual diverse mobility patterns in urban areas, as well as interactions between individuals and station of attraction, result in significant variability even among neighboring area, which is especially visible in Shenzhen's central region.And public transportation lines in relatively underdeveloped places, such as Pingshan District, have a rather uniform structure.
As seen in Figure 5 above, the results of the urban mobility structure discovery based on the interaction knowledge graph of individuals and stations and the affiliation map of stations and urban areas demonstrate that the urban mobility structure on weekends is more heterogeneous than that on weekdays, with the weekday structure being dominated by Type I and V while the weekend structure is more complex.This is mostly caused by the fact that individual travel patterns are more uniform and commute-related travel activities predominate throughout the week, whereas weekend travel activities are more diverse.The same place has diverse purposes at different times of the day.
According to the results of the individual integrated representation learning algorithm and the spatio-temporal representation algorithm of individuals and station, the inflow and outflow of Type I in the working day are low, resulting in less interaction between individuals and station, indicating that individuals are not strongly connected to the urban environment.This mobility structure is dispersed locally in the high-cost areas of Nanshan District and Futian District, demonstrating that the higher the degree of economic growth, the less eager persons are to use public transit.Type II is primarily distributed along the subway in Baoan District, Longgang District, and Longhua District, as well as along the bus route in Pingshan District, with strong accessibility.Type II is primarily distributed in areas with low and medium house prices, low density of various POIs, and Shenzhen's economic development stage is in the middle and low stage.Type III is primarily distributed in the northern part of Baoan District, the northwestern part of Pingshan District, and the edge of Longhua District, where property prices are in the middle and low levels of Shenzhen and economic development is relatively low.The areas belonging to Type III do not have subway lines passing through, making long-distance public transportation difficult.Type IV can be found on the borders of Nanshan District, Luohu District, and Futian District, as well as the northern part of Bao'an District, the junction of Guangming District and Longhua District, and Yantian District, where such mobility structures can be found on administrative district borders.Type V distribution characteristics are more visible, primarily distributed along the subway line in the more economically developed Nanshan District, Futian District, Luohu District and Longgang District, Longhua District and Baoan District, this type of urban mobility structure is located in the high housing price area, the main place of material, energy, and information exchange.
Figure 5(b) shows that, in contrast to the distribution of urban mobility structure on working days, the distribution area of Type I is obviously smaller, centered on the economic centers of Nanshan District, Luohu District, Futian District, and the northern part of Longgang District and Baoan District where the metro fiber is being developed.Type II is primarily found in Guangming District, northern Baoan District, Pingshan District, and Dapeng District, all of which are located far from the city center.House prices in the type area are in the middle and lower ranges, while economic development in Shenzhen is in the middle and lower ranges.Type III urban mobility structure is sparsely distributed, primarily in the northern portion of Baoan District near Nanshan District, Longhua District away from the metro line area, and the north-central section of Longgang District, with moderate to low property values.The Type IV is primarily found in the districts of Baoan, northwest Longhua, Guangming, and Pingshan.Nanshan District, Futian District, and Luohu District have a high level of economic development, developed public transportation, extensive leisure facilities, and a vast scale of individual inflow and outflow.However, the distribution range is significantly smaller than Type V on weekdays, owing to the fact that weekdays in such areas are primarily commuting activities, whereas on weekends residents travel for shopping, dining, and other entertainment activities, and a portion of the population does not have a strong willingness to travel on holidays; on the other hand, the southern area of Longhua District has more high-tech enterprises, which are primarily commuting activities.
To further describe the urban mobility structure on weekdays and weekends, we counted the average travel time, average stay time, average travel distance, and distance to the nearest metro station for each type of urban mobility structure, as shown in Figure 6.
According to Figure 6, the average travel time of individuals within the urban mobility structure of weekdays, except for Type IV, is greater than that of individuals within that on weekends, primarily because the traffic flow on weekdays is significantly greater than that on weekends, resulting in longer travel time for individuals within the urban mobility structure of Type I, II, III, and IV.Because the structure is mostly located in distant places, the travel time to Nanshan District, Luohu District, Futian District, and other areas for business and amusement is longer.
Looking at the average length of stay (Figure 6(b)), weekdays have a longer average length of stay than weekends, and weekday residents must spend longer at their workplace due to commuting.Individuals in weekday Type IV stay the longest, reaching more than 300 min, indicating that they travel for work-related commuting activities, indirectly indicating that the main category of Type IV may be residential places; individuals in weekend Type IV also stay longer, indicating that the urban mobility structure of this type is mostly for recreational activities such as tourism and dining.The length of stay of Type V is also longer, at about 130 min, whether on weekdays or weekends, owing to the increased demand for shopping, entertainment, and socializing among residents of Type V on weekends.It can be seen from Shenzhen city housing price data that urban mobility structure of Type V is primarily distributed in high-priced areas (about 120,000 on average).
Figure 6(c) shows that residents in Type III and Type IV have larger travel distances.Combining the spatial distribution of Type III and Type IV reveals that residents in such mobility structures primarily engage in medium and long distance commuting activities during the week, while most residents travel medium and long distances on weekends to meet leisure and entertainment needs, etc. Residents within different urban mobility structures must consequently drive longer distances on weekends to reach the core site where commercial POIs are densely located.The trip distance is the greatest for the Type IV, which is a long-distance travel pattern, owing to increased free time on weekends.Weekend Type V has the shortest travel distance and is ideal for short trips.
Figure 6(d) shows that Type I is the furthest from the nearest subway station, whether on weekdays or weekends, and because subway lines provide efficient and fast services for long-distance travel, Type I is not conducive to long-distance travel due to subway line constraints.The distances to the nearest metro stations in Type IV and V are short, making long-distance travel possible.
A flow map is constructed to summarize the deep aspects of the urban mobility structure, as shown in Figure 7.
On weekdays, there are 4,830,923 travel data points every day, and on weekends, there are 3,882,505 travel data points per day.Weekend data is 24.43% less than weekday data, indicating that individual travel intention is much lower on weekends than weekdays.
According to Figure 7(a), the distribution of various types of POIs in Type I is more uniform, with public facilities and medical care accounting for a larger share and the largest area in the area to which Type I belongs.Medical POIs in Type II account for a larger share, followed by government agencies and commercial points with smaller areas.When compared to other urban mobility structures, Type IV has the fewest POIs, indicating that the area is remote and economic  development is at a low to medium level.Type V has the most POIs, the highest proportion of public facilities, and a larger area.In terms of flow, the inflow and outflow flows between the two urban mobility structures are nearly identical, indicating that most people will return to their original locations.The flow within Type V is the largest.The closer connection between Type I and Type V and the larger flow are mainly due to the fact that the population of Type Ⅰ depends on the job opportunities, public facilities, commerce, education, etc. brought by Type V, while the flow within Type I is also large, indicating that its own resources can meet part of the basic needs.Type II is also more closely linked to Type V, followed by Type I and Type II, Type IV and Type V, with fewer links between other mobility structures.The types of POI visited by individuals with different spatial distributions are significantly different, which further leads to the differences in urban mobility structures.
Figure 7(b) demonstrates that in Type I, the distribution of POI is uniform, with the largest distribution of medical points and the smallest distribution of government agencies and public facilities with the largest area.In Type II, medical care points have the largest share and educational institutions have a smaller share.In Type III, the density of POI of each type has increased noticeably.Type V is situated in the Shenzhen economic center and has the highest density of each category of POI, with the largest distribution of government, commercial, educational, and medical points.Type IV has a sparse distribution of each category of POI, and economic development is at a low to medium level.The traffic within Type V is the largest from the perspective of each urban mobility structure, primarily because of the developed metro bus network and good accessibility.The traffic within t Type I is also significant, and the facilities within this structure are also essentially capable of meeting the basic needs of the residents.The flow between Type I and Type V is also greater, primarily because Type I and Type V are connected by subway lines, and people in Type I depend on the leisure conditions brought about by Type V, such as commerce and public facilities, and Type I have a larger number of attractions that are appealing to residents of Type V.With metro lines connecting them, the flow between Type II and Type V is also larger, making them more accessible and alluring to one another.The other urban mobility structures, with Type IV being the least connected to the other urban structures, are less connected to one another.
4.2.2.Urban mobility structure on morning peak and evening peak Urban mobility structures fluctuate greatly depending on the day of the week, as shown by the varied patterns seen on weekdays and weekends.Based on this, we continued to identify the urban mobility structure of the morning and evening peaks on weekdays (the morning peak occurs from 7:00 to 10:00, and the evening peak occurs from 17:00 to 20:00), and we examined the variable patterns in them.
Figure 8(a) morning peak map demonstrates that areas with more developed transportation infrastructure are closer to metro stations where category 1 urban mobility structure is distributed.Type 2 is more sporadically distributed, with concentrations in Baoan District along metro line 11 and Longgang District along metro line 3. Type 3 is consistent with the distribution of weekday Type 1.
Because people commute to work in the morning peak and return home in the evening peak, the urban mobility structure in the evening peak shows results that are significantly different from those in the morning peak.Since most people visit their homes during the evening peak, there are fewer structures for Type 4. The majority of Type 2 urban mobility structures are found near subway lines, which are hubs of human interaction.The morning peak distribution and the Type 3 are both consistent with the low visit area.

Comparison experiment
The following algorithms are compared in this paper in order to assess the validity of this approach: (1) Combo algorithm (Sobolevsky et al. 2014)  (4) Ablation study, spatial and temporal heterogeneity of individual residents is not considered, which focuses on mining the urban mobility structure from individual to station perspective.( 5) GraphTUL (Gao et al. 2022), considers link trajectories to users by the policy gradient to improve the identification ability.This article takes the sites that users pass through as trajectory points, but it is difficult to obtain the urban activity structure.Therefore, it is compared with the embedding prediction accuracy of this article.
The results of urban mobility structure discovery for each type of algorithm are shown in Figure 9.
Figure 9, which shows a significant localization trend, illustrates how Combo obtains a high degree of variety in urban mobility structures.The Combo algorithm can only input a single matrix and does not take into account intersection of individual and station and individual travel heterogeneity, which are crucial elements in identifying the structure of dynamic urban mobility.This is because the Combo algorithm only considers strong internal connections, which are extracted by the internal application of the modular optimization principle.
Because the GraphEncoder technique can create higher-order similarity matrices and transform topological linkages between urban areas into embeddings, it outperforms Combo in terms of showing the structure of global urban mobility structures.Similar to the Combo method, GraphEncoder only accepts a single matrix as input, making the structure of each urban mobility mines more dispersed and the distribution pattern more elusive.A deep learning-based GraphEncoder can mine the nonlinear link between the aspects of urban mobility, but it can only take the node out as a whole.It is challenging to account for the necessary travel dynamics since it can only take into consideration the node exit and entry degrees and the topological information between urban areas.Additionally, the method's inability to combine intersection information is a flaw that produces imbalanced results in the number of nodes of various urban mobility structures.GraphEncoder can only recognize basic urban mobility structures as a result.Group mobility patterns can be partially tapped into by Zhang's group trip set-based urban mobility structure discovery.The technique examines group behavior in urban areas with strong economic growth as well as group movement along subway lines.The method of fusing static and dynamic data results in a more accurate detection of the urban mobility structure since the algorithm considers both the static attribute information of the city and the dynamic travel information.However, as this method is focused on statistics of group outings, it still performs poorly when it comes to identifying more specific urban mobility patterns, such as skipping over individual interaction data in Longhua.
The individual-based integrated representational learning, as opposed to the group-based representational learning method, can reveal more intricate urban mobility structure, such as Type 8 along the subway in Baoan District and Longgang District.Since this approach considers how individual and station interact.However, this algorithm ignores the individual trip process heterogeneity, i.e. the more the individual travel process heterogeneity, the greater the distance in the representation space.
In this paper, we use the Q-value (modularity) proposed by Newman (2006) as a judgment indicator as Eq. ( 13).
A ij Represent the topological relationship between node i and node j in the entire network, when nodes i and j are directly connected, A ij = 1, conversely, A ij = 0, k i is the degree of node i, 2 * m is the degree of the entire network, s i s j is used to determine whether i and j are in the same community.And the bigger the Q-value, the more accurate the result of mobility structure discovery, as shown in Table 3.The model proposed in this paper outperforms all comparison methods on both weekdays and weekends.the Combo algorithm extracts local information and can only take into account global connections, and thus may not detect meaningful population movements.GraphEncoder is a typical deep learning clustering method that can be used for mobility structure detection.However, it only considers node degree and topological information and thus does not include the most basic population dynamics.Joint embedding does not consider the interaction between individuals and activity scenarios, making it difficult to obtain the dynamic pattern of population mobility.In contrast, our approach considers a comprehensive range of travel patterns and attributes.
This research employs the T-SNE algorithm to reduce the input data of various algorithms to 2 dimensions and visualizes the reduced 2-dimensional results, which are presented in Figure 10, in order to further compare the urban mobility structure detection outcomes of various techniques.
The results of the Combo and GraphEncoder algorithms do not display regularity, which suggests that Combo and GraphEncoder are underperforming in mining the urban mobility structure, according to the visualization results of the T-SNE method in Figure 10(a) and (b).
As seen in Figure 10(c), the results of the group travel-based approach to discovering urban mobility structure are better than Figure 10(a) and (b), primarily because it is able to consider both the travel patterns of groups and static attribute information, and the modularity constraint is enhanced during the training process, resulting in a more compact visualization.
Figure 10(d) shows that the individual representation algorithm performs well in mining some urban mobility structures, such as the Type 1, Type 4, and Type 5. From the spatial distribution, it can be deduced that the areas where these three urban mobility structures are located are visited less frequently.Whereas, it is challenging to mine the Type 7 and Type 8 located in the city center.Consequently, it can be concluded that the individual representation method performs well in simple high visit areas but underperforms in complex low visit areas.
Our method (Figure 10(d)) shows the results of urban mobility structure is superior, because the method can consider the intersection between individual and scene in detail.In addition, the spatial and temporal heterogeneity of individual residents can be taken into account, ensuring the comprehensiveness of the results.From the above figure, it can be seen that l d is between 1e-5 and 0.01, and l p is between 0.5 and 0.7.The prediction effect of site ID and distance similarity is good.This is because l d is more sensitive to distance, and the model will develop towards a direction where the difference between the predicted value and the true value is small.However, if l p is too large or too small, it will affect the model's performance.In addition, this article compared the prediction results with the Graph-TUL method, as shown in the Table 4.
From the above table, it can be seen that this article outperforms GraphTUL in terms of Precision on Station ID and distance similarity, mainly due to the different problems solved by the two.GraphTUL takes trajectory as the object to solve the prediction problem in the next step, with a short time interval.However, this article uses the site that the user passes through as the trajectory point, resulting in a longer time interval; The prediction results of the Ablation study are close to the end of the method proposed in this article, mainly due to the fact that the Ablation study also takes into account the interaction during individual travel processes, but does not consider the spatiotemporal heterogeneity of user travel, resulting in lower results than the method proposed in this article.

Conclusion and discussion
It is possible to determine the causes of these subpar transit services by analyzing typical travel patterns in particular urban mobility structures.The imbalance between work and residence in Shenzhen is specifically identified as one of the major contributors to the morning and evening peak hour directed mobility between major urban work centers and suburban single-function residential areas in the results of the urban mobility structure probe in this paper.This method can clearly find the changes of urban mobility structure on weekdays and weekends: while commuting behavior dominates the urban mobility structure on weekdays, leisure and recreational activities dominate the urban mobility structure on weekends.
The development of high-tech parks and office buildings in particular areas, such as the Longhua District, to promote general transportation accessibility is one example of how transportation community maps can be used to prioritize future land development plans in other strategic transportation planning efforts to mitigate transportation issues (Karimi, Shetab-Boushehri, and Ghadirifaraz 2019).Public transportation-oriented development should be encouraged for urban mobility structures with low public transportation travel, such the Dagong District and Yantian District, in order to increase public transportation ridership and decrease automobile use.The suggested methodology enables an in-depth comprehension of cities, including resident mobility and accessibility, socioeconomic inequities, the operation of various metropolitan districts, and the long-term performance of the current public transportation systems.Through the result graph, urban managers can have a clear understanding of the characteristics of different areas within the city, and thus provide services for bus route planning, travel time arrangement, urban construction, etc. Urban planners and managers can use this information to deliver efficient, equitable, and environmentally sustainable public services.However, there are still shortcomings in the study of urban mobility structure and further development is needed, including: (1) at different stages of model establishment, fully considering people's socio-economic factors, such as social economic status, employment, age, health conditions, etc., in order to analyze and explore the internal factors of urban activity structure from various dimensions; (2) Multiple modes of transportation should be considered, such as private cars, taxis, etc. Adding these factors can make the research results more reliable.

( 2 )
Accounting for individual travel heterogeneity: The learning process for representations takes into account the diversity in individual travel behaviors.It introduces the concept of 'generalized anisotropy' as the bedrock for renewing individual representations, accommodating the range of individual travel patterns.(3)Dynamic update of representations: Through enhancements to the learning environment within the reinforcement learning framework, the representations for individuals, stations, and urban areas undergo dynamic updates.This iterative process continually refines the acquired representations.
the collection of individuals, where |N u | is the number of individuals, v is the access relationship, P = {p i } |N p | i=1 is the collection of stations, |N p | is the amount of stations engaging with individuals.The implicit feedback between individuals and stations can be represented as Y KG u−p = {y up |u [ U, p [ P}, where y up = 1 means u and p interact and conversely y up = 0 when u and p do not interact.Station -urban area 'affiliation' knowledge graph: KG p−z = {P, b, Z}, which means that stations are 'affiliated' with urban area.Z is the set of urban areas, i.e.Z= {z i } |N z | i=1 , |N z | is the number of urban areas.

Figure 6 .
Figure 6.Statistical results.(a) Average travel time (b) Average stay time (c) Average travel distance (d) Distance to the nearest bus stop.

Figure 7 .
Figure 7. Flow between types (The size of the circle indicates the area of the urban mobility structure, the color indicates different types of urban mobility structures consistent with Figure 5, the statistical map inside the circle indicates the normalized value of the density of each type of POI, the direction of the arrow indicates the flow direction, and the thickness of the arrow indicates the flow size) (a) weekdays (b) weekends.
, one of the group-based urban mobility structure discovery algorithms, offers a general optimization framework to adapt urban mobility structure extraction with different objective functions.(2) GraphEncoder algorithm (Tian et al. 2014) clusters nodes with similar properties based on an aotu-encoder model.The input matrix for Combo and GraphEncoder is the traffic matrix between urban areas because they can only handle one network.(3) The group-based urban mobility structure discovery algorithm(Joint embedding), which is contrasted with joint representational learning (Zhang et al. 2021a).

Figure 8 .
Figure 8. Morning and evening peak urban mobility structure map (a) morning peak (b) evening peak.

Figure 9 .
Figure 9.Comparison of the algorithm's urban mobility structure on weekdays (a) Combo (b) GraphEncoder (c) Joint embedding (d) Ablation study.

4. 4 .
Parameter sensitivity analysis In order to further deepen the understanding of the model, this article provides a unified explanation of each parameter in the model, including the use of matrix diagrams to analyze the parameters sensitivity in the reward formula (1) l d and l p .The calculation of relevant indicators of Sensitivity analysis includes: 1)Precision on Station ID, Ratio ID = n s,true n s , n s,true and n s represent the correct number of station IDs and samples predicted in the test set, respectively, 2)Distance similarity, Ratio Dis = 1 |s * | , |s * | is the number of samples for the test set, dis( • ) is distance between the actual location of the station p i and the predicted location of the station pi , as shown in the Figure 11.

Figure 11 .
Figure 11.Parameter sensitivity results (a) Precision on Station ID (b) Average Similarity.

Table 1 .
Example of passenger travel chain for April 3, 2017.
collect longitude and latitude, speed, and travel direction of buses at 20-60 s intervals Transit network 8 subway lines and 808 bus lines, 10,626 subway and bus stops (99 subway stations, and 10,427 bus stops) POI geographic coordinates and category (government agencies, commercial, educational, recreational, medical services, and tourist sites)

Table 3 .
Comparison of Q-value of various methods.

Table 4 .
Comparison on Precision on Station ID and Distance similarity.