Repository navigation
pip cache always uses node process architecture, ignoring architecture key #971
Description
Description:
The pip caching code uses a cache key that does not include the Python binary's architecture. This leads to cache thrashing.
https://github.com/actions/setup-python/blob/main/src/cache-distributions/pip-cache.ts#L68-L75
These keys should include the Python binary architecture, which is specified with the architecture input to the action.
Action version:
5.2.0
Platform:
- Ubuntu
- macOS
- Windows
Runner type:
- Hosted
- Self-hosted
Tools version:
All
Activity
v-priya-kinthali commentedon Oct 28, 2024
Hello @alex 👋,
Thank you for this feature request. We will investigate it and get back to you as soon as we have some feedback.
v-aparnajyothi-y commentedon Jan 9, 2025
Hello @alex , This change, introduced in PR #896, adds the architecture to the pip cache key. This approach enhances caching efficiency by preventing cache thrashing when switching between different Python architectures (e.g., x86_64 and arm64). As a result, builds become faster, and multi-architecture workflows are better supported.
Please validate this from your end and share any further inputs to confirm the exact requirement.
alex commentedon Jan 9, 2025
No, that PR does not address this issue. If you look, you will also see that that PR was merged before I filed this.
That PR makes use of the node processes' architecture. But what I requested is the Python binary architecture.
On Windows, these can differ. On Windows, both x86 (32-bit) and x86-64 (64-bit) Python binaries can be installed, and the node process architecture will always be 64-bit.
We need to include the Python binary's architecture, which is specified with the
architecturekey to thesetup-pythonaction.v-aparnajyothi-y commentedon Jan 22, 2025
Hello @alex, Thank you for the clarification. We will review the proposed change and get back to you with feedback soon.
We appreciate your input and patience.
v-aparnajyothi-y commentedon Feb 10, 2025
Hello, Thank you for submitting this feature request and providing such a detailed explanation. After careful consideration, we’ve decided not to proceed with this change as it could potentially disrupt existing workflows, especially those relying on shared caches across different architectures. While we understand the benefits of more accurate caching, maintaining the stability of current workflows is our priority at this time.
We appreciate your suggestion and will continue to assess any future opportunities for improvement. Thank you for your understanding, and we look forward to your continued input!
alex commentedon Feb 10, 2025
I'm not sure I follow how this could disrupt existing workflows. Can you explain more?
v-aparnajyothi-y commentedon Apr 15, 2025
Hello @alex, thank you for your follow-up and for raising this important point.
Our primary concern with introducing the Python binary architecture into the cache key is the potential disruption it could cause to existing workflows—particularly those that rely on shared or pre-warmed caches across jobs or matrix configurations where Python architecture may vary. Currently, the cache key is based on the Node process architecture, allowing caches to be reused across 32-bit and 64-bit Python versions. This flexibility helps avoid redundant downloads and improves performance in workflows that benefit from cache sharing.
With the proposed change, caches would be segmented by the Python binary architecture (e.g., separate caches for x86 and x86-64), improving accuracy and preventing mismatches. However, this added granularity may lead to cache fragmentation, reduced cache reuse, and increased cold starts, which could impact performance and consistency in certain scenarios.
We appreciate this feedback as we evaluate future enhancements. Please feel free to reach out with any further questions or concerns.
Thanks again for your continued engagement!
v-aparnajyothi-y commentedon Apr 28, 2025
Hello @alex, Please let us know in case of any concerns/clarifications on the above
alex commentedon Apr 28, 2025
v-aparnajyothi-y commentedon May 21, 2025
Hello @alex, thank you for your response and for continuing the conversation.
We appreciate your perspective and understand that you may view the tradeoffs differently. While we’re currently proceeding with caution to avoid regressions in cache performance and reuse, particularly for matrix-based or shared workflows, your points around accuracy and architecture-specific behavior are well taken.
This is certainly an area we’ll continue to evaluate. We're actively monitoring how caching strategies evolve with broader usage patterns, and we'll revisit this discussion as we gather more feedback and data. Your input is valuable to that ongoing process.
Thanks again for your engagement, and please feel free to share any further thoughts or suggestions.