发布日期:2024-11-17
VIP内容
Spark Python可支持性矩阵
当Python程序员(或数据分析师)配置PySpark用于大数据分析时,该如何选择与指定版本Spark兼容的Python版本呢?或者该如何选择与指定版本的Python兼容的Spark版本呢?
Spark Python可支持性矩阵是确定哪些Python版本与特定Spark版本兼容的基本工具。这个矩阵提供了不同Spark版本中不同Python版本的兼容性级别的详细概述。
| Spark Version | Python Min Supported Version | Python Max Supported Version |
Python v 2.7 |
Python v3.4 |
Python v3.5 |
Python v 3.6 |
Python v3.7 |
Python v3.8 |
Python v3.9 |
Python v3.10 |
Python v3.11 |
| 3.5.2 | 3.8 | 3.11 | No | No | No | No | No | Yes | Yes | Yes | Yes |
| 3.5.1 | 3.8 | 3.11 | No | No | No | No | No | Yes | Yes | Yes | Yes |
| 3.5.0 | 3.8 | 3.11 | No | No | No | No | No | Yes | Yes | Yes | Yes |
| 3.4.2 | 3.7 | 3.11 | No | No | No | No | Yes | Yes | Yes | Yes | Yes |
| 3.4.1 | 3.7 | 3.11 | No | No | No | No | Yes | Yes | Yes | Yes | Yes |
| 3.4.0 | 3.7 | 3.11 | No | No | No | No | Yes | Yes | Yes | Yes | Yes |
| 3.3.3 | 3.7 | 3.10 | No | No | No | No | Yes | Yes | Yes | Yes | No |
| 3.3.2 | 3.7 | 3.10 | No | No | No | No | Yes | Yes | Yes | Yes | No |
| 3.3.1 | 3.7 | 3.10 | No | No | No | No | Yes | Yes | Yes | Yes | No |
| 3.3.0 | 3.7 | 3.10 | No | No | No | No | Yes | Yes | Yes | Yes | No |
| 3.2.4 | 3.6 | 3.9 | No | No | No | Yes | Yes | Yes | Yes | No | No |
| 3.2.3 | 3.6 | 3.9 | No | No | No | Yes | Yes | Yes | Yes | No | No |
| 3.2.2 | 3.6 | 3.9 | No | No | No | Yes | Yes | Yes | Yes | No | No |
| 3.2.1 | 3.6 | 3.9 | No | No | No | Yes | Yes | Yes | Yes | No | No |
| 3.2.0 | 3.6 | 3.9 | No | No | No | Yes | Yes | Yes | Yes | No | No |
| 3.1.3 | 3.6 | 3.9 | No | No | No | Yes | Yes | Yes | Yes | No | No |
| 3.1.2 | 3.6 | 3.9 | No | No | No | Yes | Yes | Yes | Yes | No | No |
| 3.1.1 | 3.6 | 3.9 | No | No | No | Yes | Yes | Yes | Yes | No | No |
| 3.0.3 | 2.7/3.4 | 3.9 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
| 3.0.2 | 2.7/3.4 | 3.9 | Yes | Yes | Yes | Yes | Yes | Yes | Yes | No | No |
| 3.0.1 | 2.7/3.4 | 3.8 | Yes | Yes | Yes | Yes | Yes | Yes | No | No | No |
| 3.0.0 | 2.7/3.4 | 3.8 | Yes | Yes | Yes | Yes | Yes | Yes | No | No | No |
| 3.0.0 | 2.7/3.4 | 3.8 | Yes | Yes | Yes | Yes | Yes | Yes | No | No | No |
| 3.0.0 | 2.7/3.4 | 3.8 | Yes | Yes | Yes | Yes | Yes | Yes | No | No | No |
| 2.4.8 | 2.7/3.4 | 3.7 | Yes | Yes | Yes | Yes | Yes | No | No | No | No |
| 2.4.7 | 2.7/3.4 | 3.7 | Yes | Yes | Yes | Yes | Yes | No | No | No | No |
| 2.4.6 | 2.7/3.4 | 3.7 | Yes | Yes | Yes | Yes | Yes | No | No | No | No |
| 2.4.5 | 2.7/3.4 | 3.7 | Yes | Yes | Yes | Yes | Yes | No | No | No | No |
| 2.4.4 | 2.7/3.4 | 3.7 | Yes | Yes | Yes | Yes | Yes | No | No | No | No |
| 2.4.3 | 2.7/3.4 | 3.7 | Yes | Yes | Yes | Yes | Yes | No | No | No | No |
| 2.4.2 | 2.7/3.4 | 3.7 | Yes | Yes | Yes | Yes | Yes | No | No | No | No |
| 2.4.1 | 2.7/3.4 | 3.7 | Yes | Yes | Yes | Yes | Yes | No | No | No | No |
| 2.4.0 | 2.7/3.4 | 3.7 | Yes | Yes | Yes | Yes | Yes | No | No | No | No |
| 2.3.4 | 2.7/3.4 | 3.7 | Yes | Yes | Yes | Yes | Yes | No | No | No | No |
| 2.3.3 | 2.7/3.4 | 3.7 | Yes | Yes | Yes | Yes | Yes | No | No | No | No |
| 2.3.2 | 2.7/3.4 | 3.7 | Yes | Yes | Yes | Yes | Yes | No | No | No | No |
| 2.3.1 | 2.7/3.4 | 3.6 | Yes | Yes | Yes | Yes | No | No | No | No | No |
| 2.3.0 | 2.7/3.4 | 3.6 | Yes | Yes | Yes | Yes | No | No | No | No | No |
| 2.2.3 | 2.7/3.4 | 3.5 | Yes | Yes | Yes | No | No | No | No | No | No |
| 2.2.2 | 2.7/3.4 | 3.5 | Yes | Yes | Yes | No | No | No | No | No | No |
| 2.2.1 | 2.7/3.4 | 3.5 | Yes | Yes | Yes | No | No | No | No | No | No |
| 2.2.0 | 2.7/3.4 | 3.5 | Yes | Yes | Yes | No | No | No | No | No | No |
| 2.1.3 | 2.7/3.4 | 3.5 | Yes | Yes | Yes | No | No | No | No | No | No |
| 2.1.2 | 2.7/3.4 | 3.5 | Yes | Yes | Yes | No | No | No | No | No | No |
| 2.1.1 | 2.7/3.4 | 3.5 | Yes | Yes | Yes | No | No | No | No | No | No |