ray

Problem: Currently JsonLoggerCallback.handle_result will load in the entirety of the existing results, append the new result, and then rewrite the entire file. This may not scale when running long-running jobs or jobs with large results.

https://github.com/ray-project/ray/blob/4e8f90aca20aa7bb87a4e84039889444824382ca/python/ray/train/callbacks/logging.py#L138-L142

Potential Fix:

Describe the bug

df.sort_values() failed when input dataframe is empty.

To Reproduce
To help us reproducing this bug, please provide information below:

1. Your Python version
2. The version of Mars you use
3. Versions of crucia

Overview

As a CFP user, I would like to split a dataset (e.g., np array, pandas dataframe) into smaller objects that can then be fed into other nodes/pipeline. This is especially useful when we have compute intensive tasks and would like to parallelize it eas

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

Currently, we only lint & test ray-operator, since we merge service code here, we should also update CI to support these submodules.

https://github.com/ray-project/kuberay/blob/6f87ca64c107cd51d3ab955faf4be198e0094536/.git