Hi, I'm facing some issues with Ray when running it locally on my mac (M1, Sonoma 14.1.1). I wonder whether for those of us running locally, a different configuration or setup of Ray would be required.
Many thanks
2024-03-08 12:31:00,522 ERROR services.py:1222 -- Failed to start the dashboard , return code 1
2024-03-08 12:31:00,524 ERROR services.py:1247 -- Error should be written to 'dashboard.log' or 'dashboard.err'. We are printing the last 20 lines for you. See 'https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure' to find where the log file is.
2024-03-08 12:31:00,529 ERROR services.py:1291 --
The last 20 lines of /tmp/ray/session_2024-03-08_12-30-58_875047_6306/logs/dashboard.log (it contains the error message from the dashboard):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/rubencontesti/Code/madewithml/venv/lib/python3.11/site-packages/ray/dashboard/utils.py", line 134, in get_all_modules
raise e
File "/Users/rubencontesti/Code/madewithml/venv/lib/python3.11/site-packages/ray/dashboard/utils.py", line 121, in get_all_modules
importlib.import_module(name)
File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/Users/rubencontesti/Code/madewithml/venv/lib/python3.11/site-packages/ray/dashboard/modules/test/test_agent.py", line 6, in <module>
import ray.dashboard.modules.test.test_utils as test_utils
File "/Users/rubencontesti/Code/madewithml/venv/lib/python3.11/site-packages/ray/dashboard/modules/test/test_utils.py", line 3, in <module>
import async_timeout
ModuleNotFoundError: No module named 'async_timeout'
2024-03-08 12:31:00,654 INFO worker.py:1642 -- Started a local Ray instance.
(raylet) [2024-03-08 12:31:01,402 E 6333 155252] (raylet) agent_manager.cc:70: The raylet exited immediately because one Ray agent failed, agent_name = dashboard_agent/470211272.
(raylet) The raylet fate shares with the agent. This can happen because
(raylet) - The version of `grpcio` doesn't follow Ray's requirement. Agent can segfault with the incorrect `grpcio` version. Check the grpcio version `pip freeze | grep grpcio`.
(raylet) - The agent failed to start because of unexpected error or port conflict. Read the log `cat /tmp/ray/session_latest/logs/{dashboard_agent|runtime_env_agent}.log`. You can find the log file structure here https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure.
(raylet) - The agent is killed by the OS (e.g., out of memory).
Originally created by @rcontesti on GitHub (Mar 8, 2024).
Original GitHub issue: https://github.com/GokuMohandas/Made-With-ML/issues/255
Hi, I'm facing some issues with Ray when running it locally on my mac (M1, Sonoma 14.1.1). I wonder whether for those of us running locally, a different configuration or setup of Ray would be required.
Many thanks
```
2024-03-08 12:31:00,522 ERROR services.py:1222 -- Failed to start the dashboard , return code 1
2024-03-08 12:31:00,524 ERROR services.py:1247 -- Error should be written to 'dashboard.log' or 'dashboard.err'. We are printing the last 20 lines for you. See 'https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure' to find where the log file is.
2024-03-08 12:31:00,529 ERROR services.py:1291 --
The last 20 lines of /tmp/ray/session_2024-03-08_12-30-58_875047_6306/logs/dashboard.log (it contains the error message from the dashboard):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/rubencontesti/Code/madewithml/venv/lib/python3.11/site-packages/ray/dashboard/utils.py", line 134, in get_all_modules
raise e
File "/Users/rubencontesti/Code/madewithml/venv/lib/python3.11/site-packages/ray/dashboard/utils.py", line 121, in get_all_modules
importlib.import_module(name)
File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 940, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/Users/rubencontesti/Code/madewithml/venv/lib/python3.11/site-packages/ray/dashboard/modules/test/test_agent.py", line 6, in <module>
import ray.dashboard.modules.test.test_utils as test_utils
File "/Users/rubencontesti/Code/madewithml/venv/lib/python3.11/site-packages/ray/dashboard/modules/test/test_utils.py", line 3, in <module>
import async_timeout
ModuleNotFoundError: No module named 'async_timeout'
2024-03-08 12:31:00,654 INFO worker.py:1642 -- Started a local Ray instance.
(raylet) [2024-03-08 12:31:01,402 E 6333 155252] (raylet) agent_manager.cc:70: The raylet exited immediately because one Ray agent failed, agent_name = dashboard_agent/470211272.
(raylet) The raylet fate shares with the agent. This can happen because
(raylet) - The version of `grpcio` doesn't follow Ray's requirement. Agent can segfault with the incorrect `grpcio` version. Check the grpcio version `pip freeze | grep grpcio`.
(raylet) - The agent failed to start because of unexpected error or port conflict. Read the log `cat /tmp/ray/session_latest/logs/{dashboard_agent|runtime_env_agent}.log`. You can find the log file structure here https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure.
(raylet) - The agent is killed by the OS (e.g., out of memory).
```
Leaving this for anyone else experiencing this. in requirements.txt I changed the ray[air]==2.7.0 to ray[default]==2.7.0 and it works.
<!-- gh-comment-id:2543831456 -->
@divakaivan commented on GitHub (Dec 15, 2024):
Leaving this for anyone else experiencing this. in `requirements.txt` I changed the `ray[air]==2.7.0` to `ray[default]==2.7.0` and it works.
thank you so much! I'll probably working on my ray self-hosted ml platform soon :D
<!-- gh-comment-id:2549586730 -->
@mertbozkir commented on GitHub (Dec 17, 2024):
thank you so much! I'll probably working on my ray self-hosted ml platform soon :D
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @rcontesti on GitHub (Mar 8, 2024).
Original GitHub issue: https://github.com/GokuMohandas/Made-With-ML/issues/255
Hi, I'm facing some issues with Ray when running it locally on my mac (M1, Sonoma 14.1.1). I wonder whether for those of us running locally, a different configuration or setup of Ray would be required.
Many thanks
@divakaivan commented on GitHub (Dec 15, 2024):
Leaving this for anyone else experiencing this. in
requirements.txtI changed theray[air]==2.7.0toray[default]==2.7.0and it works.@mertbozkir commented on GitHub (Dec 17, 2024):
thank you so much! I'll probably working on my ray self-hosted ml platform soon :D