Autonomous spacecraft control in the solar gravitational lens' focus via reinforcement learning

Cover Page

Cite item

Full Text

Open Access Open Access
Restricted Access Access granted
Restricted Access Subscription Access

Abstract

The problem of autonomous control of the translational motion of the spacecraft in the vicinity of the focus of the gravitational lens of the Sun is formulated. The problem is solved by a reinforcement machine learning method using contemporary stochastic numerical methods. The costs of the characteristic velocity for targeting the focal line of a remote extended source, the final accuracy of targeting and the quality of the control function are investigated. The results of the study are given for various forms of state and observation: 1) position and velocity, 2) noisy position and velocity, 3) image of the Einstein ring. The efficiency of control strategies when using recurrent layers and fully connected layers with an input in the form of a measurement stack is compared. The training of control models accounting for execution errors of maneuvers is also being explored.

Full Text

Restricted Access

About the authors

M. G. Shirobokov

Keldysh Institute of Applied Mathematics

Author for correspondence.
Email: shirobokov@keldysh.ru
Russian Federation, Miusskaya Pl., 4, Moscow

K. R. Korneev

Keldysh Institute of Applied Mathematics

Email: shirobokov@keldysh.ru
Russian Federation, Miusskaya Pl., 4, Moscow

D. G. Perepukhov

Keldysh Institute of Applied Mathematics

Email: shirobokov@keldysh.ru
Russian Federation, Miusskaya Pl., 4, Moscow

References

  1. Brandt P.C., Provornikova E.A., Cocoros A. et al. Interstellar Probe: Humanity’s exploration of the Galaxy Begins // Acta Astronautica. 2022. V. 199. P. 364–373. https://doi.org/10.1016/j.actaastro.2022.07.011
  2. Einstein A. The Field Equations of Gravitation // Preussische Akademie der Wissenschaften, Sitzungsberichte, (Math. Phys.). Berlin, 1915. P. 844–847.
  3. Eddington A.S. Space, time and gravitation: An outline of the general relativity theory. Cambridge University Press, 1920.
  4. Фок В.А. Теория пространства, времени и тяготения. Москва: Физматгиз, 1955.
  5. Turyshev S.G., Toth V.T. Resolved imaging of exoplanets with the solar gravitational lens // Monthly Notices of the Royal Astronomical Society. 2022. V. 515. Iss. 4. P. 6122–6132. https://doi.org/10.1093/mnras/stac2130
  6. Turyshev S.G. Wave-theoretical description of the solar gravitational lens // Physical Review. 2017. V. 95. Iss. 8. Art. ID. 084041. https://doi.org/10.1103/PhysRevD.95.084041
  7. Turyshev S.G., Toth V.T. Wave-optical treatment of the shadow cast by a large gravitating sphere // Physical Review. 2018. V. 98. Iss. 10. Art. ID. 104015. https://doi.org/10.1103/PhysRevD.98.104015
  8. Turyshev S.G., Toth V.T. Optical properties of the solar gravitational lens in the presence of the solar corona // European Physical J. Plus. 2019. V. 134. Art. ID. 63. https://doi.org/10.1140/epjp/i2019-12426-4
  9. Turyshev S.G., Toth V.T. Image formation for extended sources with the solar gravitational lens // Physical Review. 2020. V. 102. Iss. 2. Art. ID. 024038. https://doi.org/10.1103/PhysRevD.102.024038
  10. Toth V.T., Turyshev S.G. Image recovery with the solar gravitational lens // Physical Review. 2021. V. 103. Iss. 12. Art. ID. 124038. https://doi.org/10.1103/PhysRevD.103.124038
  11. Willems P.A. Photometric Limits on the High Resolution Imaging of Exoplanets Using the Solar Gravity Lens // Acta Astronautica. 2018. V. 152. P. 408–414. https://doi.org/10.1016/j.actaastro.2018.08.013
  12. Turyshev S.G., Shao M., Alkalai L. et al. Direct Multipixel Imaging and Spectroscopy of an exoplanet with a Solar Gravity Lens Mission // Final Report. NASA Innovative Advanced Concepts (NIAC). Phase I. 2018. https://arxiv.org/abs/1802.08421
  13. Turyshev S.G., Shao M., Toth V.T. et al. Direct Multipixel Imaging and Spectroscopy of an Exoplanet with a Solar Gravity Lens Mission // Final Report. NASA Innovative Advanced Concepts (NIAC). Phase II. 2020. https://arxiv.org/abs/2002.11871
  14. Саттон Р.С., Барто Э.Г. Обучение с подкреплением. Москва: Бином. Лаборатория знаний, 2017.
  15. Bertsekas D.P. Reinforcement learning and optimal control. Belmont: Athena Scientific, 2019.
  16. Kamalapurkar R., Walters P., Rosenfeld J. et al. Reinforcement Learning for Optimal Feedback Control. A Lyapunov-Based Approach. Cham: Springer, 2018.
  17. Shirobokov M., Trofimov S., Ovchinnikov M. Survey of machine learning techniques in spacecraft control design // Acta Astronautica. 2021. V. 186. P. 87–97. https://doi.org/10.1016/j.actaastro.2021.05.018
  18. Gaudet B., Linares R., Furfaro R. Terminal adaptive guidance via reinforcement meta-learning: Applications to autonomous asteroid close-proximity operations // Acta Astronautica. 2020. V. 171. P. 1–13. https://doi.org/10.1016/j.actaastro.2020.02.036
  19. Gaudet B., Linares R., Furfaro R. Adaptive guidance and integrated navigation with reinforcement meta-learning // Acta Astronautica. 2020. V. 169. P. 180–190. https://doi.org/10.1016/j.actaastro.2020.01.007
  20. Scorsoglio A., D’Ambrosio A., Ghilardi L. et al. Image-based deep reinforcement meta-learning for autonomous lunar landing // J. Spacecraft and Rockets. 2022. V. 59. Iss. 1. P. 153–165. https://doi.org/10.2514/1.A35072
  21. Gaudet B., Linares R., Furfaro R. Six degree-of-freedom body-fixed hovering over unmapped asteroids via LIDAR altimetry and reinforcement meta-learning // Acta Astronautica. 2020. V. 172. P. 90–99. https://doi.org/10.1016/j.actaastro.2020.03.026
  22. Широбоков М.Г. Методика построения управления космическими аппаратами с использованием методов обучения с подкреплением // Косм. исслед. 2024. Т. 62. № 5. С. 498–515. https://doi.org/10.31857/S0023420624050082
  23. Lefor A.T., Futamase T., Akhlaghi M. A systematic review of strong gravitational lens modeling software // New Astronomy Reviews. 2013. V. 57. Iss. 1–2. P. 1–13. https://doi.org/10.1016/j.newar.2013.05.001
  24. Oguri M. The Mass Distribution of SDSS J1004+4112 Revisited // Public. Astronomical Society of Japan. 2010. V. 62. Iss. 4. P. 1017–1024. https://doi.org/10.1093/pasj/62.4.1017
  25. Silver D., Lever G., Heess N. et al. Deterministic policy gradient algorithms // Proc. 31st  Intern. Conf. Machine Learning. Beijing, China. 2014. V. 32. Iss. 1. P. 387–395. http://proceedings.mlr.press/v32/silver14.html
  26. Mnih V., Badia A.P., Mirza M. et al. Asynchronous Methods for Deep Reinforcement Learning // Proc. 33rd Intern. Conf. Machine Learning. New York, USA. 2016. V. 48. P. 1928–1937. https://proceedings.mlr.press/v48/mniha16.html
  27. Schulman J., Wolski F., Dhariwal P. et al. Proximal Policy Optimization Algorithms // arXiv preprint. 2017. Art. ID. 1707.06347. https://arxiv.org/abs/1707.06347
  28. Moriarty D.E., Schultz A. C., Grefenstette J.J. Evolutionary algorithms for reinforcement learning // J. Artificial Intelligence Research. 1999. V. 11. P. 241–276.
  29. Sehgal A., La H., Louis S. et al. Deep reinforcement learning using genetic algorithm for parameter optimization // Proc. Third IEEE International Conference on Robotic Computing (IRC). 2019. P. 596–601. https://doi.org/10.1109/IRC.2019.00121
  30. Hochreiter S., Schmidhuber J. Long short-term memory // Neural computation. 1997. V. 9. Iss. 8. P. 1735–1780.
  31. Hoeffding W. Probability inequalities for sums of bounded random variables // J. American Statistical Association. 1963. V. 58. Iss. 301. P. 13–30. https://doi.org/10.1080/01621459.1963.10500830

Supplementary files

Supplementary Files
Action
1. JATS XML
Download
2. Fig. 1. Schematic representation of the curvature of light rays from a distant source under the influence of a solar gravitational lens.

Download (128KB)
Indexing metadata
3. Fig. 2. Image formed by glafic2 for the spacecraft at a distance of 130 thousand km from the focal line and at a distance of 600 AU from the Sun.

Download (34KB)
Indexing metadata
4. Fig. 3. Image formed by glafic2 for the spacecraft at a distance of 85 thousand km from the focal line and at a distance of 600 AU from the Sun.

Download (40KB)
Indexing metadata
5. Fig. 4. Initial and final distances (blue dots) to the focal line obtained in a series of Monte Carlo trials for the strategy based on the state of the vehicle; the line of equality of distances is marked in red.

Download (151KB)
Indexing metadata
6. Fig. 5. Initial and final distances (blue dots) to the focal line obtained in a series of Monte Carlo tests for the strategy based on the state of the vehicle.

Download (196KB)
Indexing metadata
7. Fig. 6. Initial and final distances (blue dots) to the focal line obtained in a series of Monte Carlo trials for the image stack-based strategy.

Download (216KB)
Indexing metadata

Copyright (c) 2025 Russian Academy of Sciences