Skip to content

Commit e33200b

Browse files
davem330davem330
committed
Merge branch 'tls-offload-netdev-and-mlx5-support'
Boris Pismenny says: ==================== TLS offload, netdev & MLX5 support The following series provides TLS TX inline crypto offload. v1->v2: - Added IS_ENABLED(CONFIG_TLS_DEVICE) and a STATIC_KEY for icsk_clean_acked - File license fix - Fix spelling, comment by DaveW - Move memory allocations out of tls_set_device_offload and other misc fixes, comments by Kiril. v2->v3: - Reversed xmas tree where needed and style fixes - Removed the need for skb_page_frag_refill, per Eric's comment - IPv6 dependency fixes v3->v4: - Remove "inline" from functions in C files - Make clean_acked_data_enabled a static variable and add enable/disable functions to control it. - Remove unnecessary variable initialization mentioned by ShannonN - Rebase over TLS RX - Refactor the tls_software_fallback to reduce the number of variables mentioned by KirilT v4->v5: - Add missing CONFIG_TLS_DEVICE v5->v6: - Move changes to the software implementation into a seperate patch - Fix some checkpatch warnings - GPL export the enable/disable clean_acked_data functions v6->v7: - Use the dst_entry to obtain the netdev in dev_get_by_index - Remove the IPv6 patch since it is redundent now v7->v8: - Fix a merge conflict in mlx5 header v8->v9: - Fix false -Wmaybe-uninitialized warning - Fix empty space in the end of new files v9->v10: - Remove default "n" in net/Kconfig This series adds a generic infrastructure to offload TLS crypto to a network devices. It enables the kernel TLS socket to skip encryption and authentication operations on the transmit side of the data path. Leaving those computationally expensive operations to the NIC. The NIC offload infrastructure builds TLS records and pushes them to the TCP layer just like the SW KTLS implementation and using the same API. TCP segmentation is mostly unaffected. Currently the only exception is that we prevent mixed SKBs where only part of the payload requires offload. In the future we are likely to add a similar restriction following a change cipher spec record. The notable differences between SW KTLS and NIC offloaded TLS implementations are as follows: 1. The offloaded implementation builds "plaintext TLS record", those records contain plaintext instead of ciphertext and place holder bytes instead of authentication tags. 2. The offloaded implementation maintains a mapping from TCP sequence number to TLS records. Thus given a TCP SKB sent from a NIC offloaded TLS socket, we can use the tls NIC offload infrastructure to obtain enough context to encrypt the payload of the SKB. A TLS record is released when the last byte of the record is ack'ed, this is done through the new icsk_clean_acked callback. The infrastructure should be extendable to support various NIC offload implementations. However it is currently written with the implementation below in mind: The NIC assumes that packets from each offloaded stream are sent as plaintext and in-order. It keeps track of the TLS records in the TCP stream. When a packet marked for offload is transmitted, the NIC encrypts the payload in-place and puts authentication tags in the relevant place holders. The responsibility for handling out-of-order packets (i.e. TCP retransmission, qdisc drops) falls on the netdev driver. The netdev driver keeps track of the expected TCP SN from the NIC's perspective. If the next packet to transmit matches the expected TCP SN, the driver advances the expected TCP SN, and transmits the packet with TLS offload indication. If the next packet to transmit does not match the expected TCP SN. The driver calls the TLS layer to obtain the TLS record that includes the TCP of the packet for transmission. Using this TLS record, the driver posts a work entry on the transmit queue to reconstruct the NIC TLS state required for the offload of the out-of-order packet. It updates the expected TCP SN accordingly and transmit the now in-order packet. The same queue is used for packet transmission and TLS context reconstruction to avoid the need for flushing the transmit queue before issuing the context reconstruction request. Expected TCP SN is accessed without a lock, under the assumption that TCP doesn't transmit SKBs from different TX queue concurrently. If packets are rerouted to a different netdevice, then a software fallback routine handles encryption. Paper: https://www.netdevconf.org/1.2/papers/netdevconf-TLS.pdf ==================== Signed-off-by: David S. Miller <[email protected]>
2 parents 1a1f4a2 + f9c8141 commit e33200b

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

43 files changed

+3355
-191
lines changed

‎MAINTAINERS

Lines changed: 5 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -9037,26 +9037,17 @@ W: http://www.mellanox.com
90379037
Q: http://patchwork.ozlabs.org/project/netdev/list/
90389038
F: drivers/net/ethernet/mellanox/mlx5/core/en_*
90399039

9040-
MELLANOX ETHERNET INNOVA DRIVER
9041-
M: Ilan Tayari <[email protected]>
9042-
R: Boris Pismenny <[email protected]>
9040+
MELLANOX ETHERNET INNOVA DRIVERS
9041+
M: Boris Pismenny <[email protected]>
90439042
L: [email protected]
90449043
S: Supported
90459044
W: http://www.mellanox.com
90469045
Q: http://patchwork.ozlabs.org/project/netdev/list/
9046+
F: drivers/net/ethernet/mellanox/mlx5/core/en_accel/*
9047+
F: drivers/net/ethernet/mellanox/mlx5/core/accel/*
90479048
F: drivers/net/ethernet/mellanox/mlx5/core/fpga/*
90489049
F: include/linux/mlx5/mlx5_ifc_fpga.h
90499050

9050-
MELLANOX ETHERNET INNOVA IPSEC DRIVER
9051-
M: Ilan Tayari <[email protected]>
9052-
R: Boris Pismenny <[email protected]>
9053-
L: [email protected]
9054-
S: Supported
9055-
W: http://www.mellanox.com
9056-
Q: http://patchwork.ozlabs.org/project/netdev/list/
9057-
F: drivers/net/ethernet/mellanox/mlx5/core/en_ipsec/*
9058-
F: drivers/net/ethernet/mellanox/mlx5/core/ipsec*
9059-
90609051
MELLANOX ETHERNET SWITCH DRIVERS
90619052
M: Jiri Pirko <[email protected]>
90629053
M: Ido Schimmel <[email protected]>
@@ -9848,7 +9839,7 @@ F: net/netfilter/xt_CONNSECMARK.c
98489839
F: net/netfilter/xt_SECMARK.c
98499840

98509841
NETWORKING [TLS]
9851-
M: Ilya Lesokhin <ilyal@mellanox.com>
9842+
M: Boris Pismenny <borisp@mellanox.com>
98529843
M: Aviad Yehezkel <[email protected]>
98539844
M: Dave Watson <[email protected]>
98549845
L: [email protected]

‎drivers/net/ethernet/mellanox/mlx5/core/Kconfig

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,3 +86,14 @@ config MLX5_EN_IPSEC
8686
Build support for IPsec cryptography-offload accelaration in the NIC.
8787
Note: Support for hardware with this capability needs to be selected
8888
for this option to become available.
89+
90+
config MLX5_EN_TLS
91+
bool "TLS cryptography-offload accelaration"
92+
depends on MLX5_CORE_EN
93+
depends on TLS_DEVICE
94+
depends on MLX5_ACCEL
95+
default n
96+
---help---
97+
Build support for TLS cryptography-offload accelaration in the NIC.
98+
Note: Support for hardware with this capability needs to be selected
99+
for this option to become available.

‎drivers/net/ethernet/mellanox/mlx5/core/Makefile

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,10 @@ mlx5_core-y := main.o cmd.o debugfs.o fw.o eq.o uar.o pagealloc.o \
88
fs_counters.o rl.o lag.o dev.o wq.o lib/gid.o lib/clock.o \
99
diag/fs_tracepoint.o
1010

11-
mlx5_core-$(CONFIG_MLX5_ACCEL) += accel/ipsec.o
11+
mlx5_core-$(CONFIG_MLX5_ACCEL) += accel/ipsec.o accel/tls.o
1212

1313
mlx5_core-$(CONFIG_MLX5_FPGA) += fpga/cmd.o fpga/core.o fpga/conn.o fpga/sdk.o \
14-
fpga/ipsec.o
14+
fpga/ipsec.o fpga/tls.o
1515

1616
mlx5_core-$(CONFIG_MLX5_CORE_EN) += en_main.o en_common.o en_fs.o en_ethtool.o \
1717
en_tx.o en_rx.o en_dim.o en_txrx.o en_stats.o vxlan.o \
@@ -28,4 +28,6 @@ mlx5_core-$(CONFIG_MLX5_CORE_IPOIB) += ipoib/ipoib.o ipoib/ethtool.o ipoib/ipoib
2828
mlx5_core-$(CONFIG_MLX5_EN_IPSEC) += en_accel/ipsec.o en_accel/ipsec_rxtx.o \
2929
en_accel/ipsec_stats.o
3030

31+
mlx5_core-$(CONFIG_MLX5_EN_TLS) += en_accel/tls.o en_accel/tls_rxtx.o en_accel/tls_stats.o
32+
3133
CFLAGS_tracepoint.o := -I$(src)

‎drivers/net/ethernet/mellanox/mlx5/core/accel/tls.c

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
/*
2+
* Copyright (c) 2018 Mellanox Technologies. All rights reserved.
3+
*
4+
* This software is available to you under a choice of one of two
5+
* licenses. You may choose to be licensed under the terms of the GNU
6+
* General Public License (GPL) Version 2, available from the file
7+
* COPYING in the main directory of this source tree, or the
8+
* OpenIB.org BSD license below:
9+
*
10+
* Redistribution and use in source and binary forms, with or
11+
* without modification, are permitted provided that the following
12+
* conditions are met:
13+
*
14+
* - Redistributions of source code must retain the above
15+
* copyright notice, this list of conditions and the following
16+
* disclaimer.
17+
*
18+
* - Redistributions in binary form must reproduce the above
19+
* copyright notice, this list of conditions and the following
20+
* disclaimer in the documentation and/or other materials
21+
* provided with the distribution.
22+
*
23+
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
24+
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
25+
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
26+
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
27+
* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
28+
* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
29+
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
30+
* SOFTWARE.
31+
*
32+
*/
33+
34+
#include <linux/mlx5/device.h>
35+
36+
#include "accel/tls.h"
37+
#include "mlx5_core.h"
38+
#include "fpga/tls.h"
39+
40+
int mlx5_accel_tls_add_tx_flow(struct mlx5_core_dev *mdev, void *flow,
41+
struct tls_crypto_info *crypto_info,
42+
u32 start_offload_tcp_sn, u32 *p_swid)
43+
{
44+
return mlx5_fpga_tls_add_tx_flow(mdev, flow, crypto_info,
45+
start_offload_tcp_sn, p_swid);
46+
}
47+
48+
void mlx5_accel_tls_del_tx_flow(struct mlx5_core_dev *mdev, u32 swid)
49+
{
50+
mlx5_fpga_tls_del_tx_flow(mdev, swid, GFP_KERNEL);
51+
}
52+
53+
bool mlx5_accel_is_tls_device(struct mlx5_core_dev *mdev)
54+
{
55+
return mlx5_fpga_is_tls_device(mdev);
56+
}
57+
58+
u32 mlx5_accel_tls_device_caps(struct mlx5_core_dev *mdev)
59+
{
60+
return mlx5_fpga_tls_device_caps(mdev);
61+
}
62+
63+
int mlx5_accel_tls_init(struct mlx5_core_dev *mdev)
64+
{
65+
return mlx5_fpga_tls_init(mdev);
66+
}
67+
68+
void mlx5_accel_tls_cleanup(struct mlx5_core_dev *mdev)
69+
{
70+
mlx5_fpga_tls_cleanup(mdev);
71+
}

‎drivers/net/ethernet/mellanox/mlx5/core/accel/tls.h

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
/*
2+
* Copyright (c) 2018 Mellanox Technologies. All rights reserved.
3+
*
4+
* This software is available to you under a choice of one of two
5+
* licenses. You may choose to be licensed under the terms of the GNU
6+
* General Public License (GPL) Version 2, available from the file
7+
* COPYING in the main directory of this source tree, or the
8+
* OpenIB.org BSD license below:
9+
*
10+
* Redistribution and use in source and binary forms, with or
11+
* without modification, are permitted provided that the following
12+
* conditions are met:
13+
*
14+
* - Redistributions of source code must retain the above
15+
* copyright notice, this list of conditions and the following
16+
* disclaimer.
17+
*
18+
* - Redistributions in binary form must reproduce the above
19+
* copyright notice, this list of conditions and the following
20+
* disclaimer in the documentation and/or other materials
21+
* provided with the distribution.
22+
*
23+
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
24+
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
25+
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
26+
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
27+
* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
28+
* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
29+
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
30+
* SOFTWARE.
31+
*
32+
*/
33+
34+
#ifndef __MLX5_ACCEL_TLS_H__
35+
#define __MLX5_ACCEL_TLS_H__
36+
37+
#include <linux/mlx5/driver.h>
38+
#include <linux/tls.h>
39+
40+
#ifdef CONFIG_MLX5_ACCEL
41+
42+
enum {
43+
MLX5_ACCEL_TLS_TX = BIT(0),
44+
MLX5_ACCEL_TLS_RX = BIT(1),
45+
MLX5_ACCEL_TLS_V12 = BIT(2),
46+
MLX5_ACCEL_TLS_V13 = BIT(3),
47+
MLX5_ACCEL_TLS_LRO = BIT(4),
48+
MLX5_ACCEL_TLS_IPV6 = BIT(5),
49+
MLX5_ACCEL_TLS_AES_GCM128 = BIT(30),
50+
MLX5_ACCEL_TLS_AES_GCM256 = BIT(31),
51+
};
52+
53+
struct mlx5_ifc_tls_flow_bits {
54+
u8 src_port[0x10];
55+
u8 dst_port[0x10];
56+
union mlx5_ifc_ipv6_layout_ipv4_layout_auto_bits src_ipv4_src_ipv6;
57+
union mlx5_ifc_ipv6_layout_ipv4_layout_auto_bits dst_ipv4_dst_ipv6;
58+
u8 ipv6[0x1];
59+
u8 direction_sx[0x1];
60+
u8 reserved_at_2[0x1e];
61+
};
62+
63+
int mlx5_accel_tls_add_tx_flow(struct mlx5_core_dev *mdev, void *flow,
64+
struct tls_crypto_info *crypto_info,
65+
u32 start_offload_tcp_sn, u32 *p_swid);
66+
void mlx5_accel_tls_del_tx_flow(struct mlx5_core_dev *mdev, u32 swid);
67+
bool mlx5_accel_is_tls_device(struct mlx5_core_dev *mdev);
68+
u32 mlx5_accel_tls_device_caps(struct mlx5_core_dev *mdev);
69+
int mlx5_accel_tls_init(struct mlx5_core_dev *mdev);
70+
void mlx5_accel_tls_cleanup(struct mlx5_core_dev *mdev);
71+
72+
#else
73+
74+
static inline int
75+
mlx5_accel_tls_add_tx_flow(struct mlx5_core_dev *mdev, void *flow,
76+
struct tls_crypto_info *crypto_info,
77+
u32 start_offload_tcp_sn, u32 *p_swid) { return 0; }
78+
static inline void mlx5_accel_tls_del_tx_flow(struct mlx5_core_dev *mdev, u32 swid) { }
79+
static inline bool mlx5_accel_is_tls_device(struct mlx5_core_dev *mdev) { return false; }
80+
static inline u32 mlx5_accel_tls_device_caps(struct mlx5_core_dev *mdev) { return 0; }
81+
static inline int mlx5_accel_tls_init(struct mlx5_core_dev *mdev) { return 0; }
82+
static inline void mlx5_accel_tls_cleanup(struct mlx5_core_dev *mdev) { }
83+
84+
#endif
85+
86+
#endif /* __MLX5_ACCEL_TLS_H__ */

‎drivers/net/ethernet/mellanox/mlx5/core/en.h

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,9 @@
5555

5656
struct page_pool;
5757

58+
#define MLX5E_METADATA_ETHER_TYPE (0x8CE4)
59+
#define MLX5E_METADATA_ETHER_LEN 8
60+
5861
#define MLX5_SET_CFG(p, f, v) MLX5_SET(create_flow_group_in, p, f, v)
5962

6063
#define MLX5E_ETH_HARD_MTU (ETH_HLEN + VLAN_HLEN + ETH_FCS_LEN)
@@ -332,6 +335,7 @@ enum {
332335
MLX5E_SQ_STATE_RECOVERING,
333336
MLX5E_SQ_STATE_IPSEC,
334337
MLX5E_SQ_STATE_AM,
338+
MLX5E_SQ_STATE_TLS,
335339
};
336340

337341
struct mlx5e_sq_wqe_info {
@@ -797,6 +801,9 @@ struct mlx5e_priv {
797801
#ifdef CONFIG_MLX5_EN_IPSEC
798802
struct mlx5e_ipsec *ipsec;
799803
#endif
804+
#ifdef CONFIG_MLX5_EN_TLS
805+
struct mlx5e_tls *tls;
806+
#endif
800807
};
801808

802809
struct mlx5e_profile {
@@ -827,6 +834,8 @@ void mlx5e_build_ptys2ethtool_map(void);
827834
u16 mlx5e_select_queue(struct net_device *dev, struct sk_buff *skb,
828835
void *accel_priv, select_queue_fallback_t fallback);
829836
netdev_tx_t mlx5e_xmit(struct sk_buff *skb, struct net_device *dev);
837+
netdev_tx_t mlx5e_sq_xmit(struct mlx5e_txqsq *sq, struct sk_buff *skb,
838+
struct mlx5e_tx_wqe *wqe, u16 pi);
830839

831840
void mlx5e_completion_event(struct mlx5_core_cq *mcq);
832841
void mlx5e_cq_error_event(struct mlx5_core_cq *mcq, enum mlx5_event event);
@@ -942,6 +951,18 @@ static inline bool mlx5e_tunnel_inner_ft_supported(struct mlx5_core_dev *mdev)
942951
MLX5_CAP_FLOWTABLE_NIC_RX(mdev, ft_field_support.inner_ip_version));
943952
}
944953

954+
static inline void mlx5e_sq_fetch_wqe(struct mlx5e_txqsq *sq,
955+
struct mlx5e_tx_wqe **wqe,
956+
u16 *pi)
957+
{
958+
struct mlx5_wq_cyc *wq;
959+
960+
wq = &sq->wq;
961+
*pi = sq->pc & wq->sz_m1;
962+
*wqe = mlx5_wq_cyc_get_wqe(wq, *pi);
963+
memset(*wqe, 0, sizeof(**wqe));
964+
}
965+
945966
static inline
946967
struct mlx5e_tx_wqe *mlx5e_post_nop(struct mlx5_wq_cyc *wq, u32 sqn, u16 *pc)
947968
{

‎drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
/*
2+
* Copyright (c) 2018 Mellanox Technologies. All rights reserved.
3+
*
4+
* This software is available to you under a choice of one of two
5+
* licenses. You may choose to be licensed under the terms of the GNU
6+
* General Public License (GPL) Version 2, available from the file
7+
* COPYING in the main directory of this source tree, or the
8+
* OpenIB.org BSD license below:
9+
*
10+
* Redistribution and use in source and binary forms, with or
11+
* without modification, are permitted provided that the following
12+
* conditions are met:
13+
*
14+
* - Redistributions of source code must retain the above
15+
* copyright notice, this list of conditions and the following
16+
* disclaimer.
17+
*
18+
* - Redistributions in binary form must reproduce the above
19+
* copyright notice, this list of conditions and the following
20+
* disclaimer in the documentation and/or other materials
21+
* provided with the distribution.
22+
*
23+
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
24+
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
25+
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
26+
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
27+
* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
28+
* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
29+
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
30+
* SOFTWARE.
31+
*
32+
*/
33+
34+
#ifndef __MLX5E_EN_ACCEL_H__
35+
#define __MLX5E_EN_ACCEL_H__
36+
37+
#ifdef CONFIG_MLX5_ACCEL
38+
39+
#include <linux/skbuff.h>
40+
#include <linux/netdevice.h>
41+
#include "en_accel/ipsec_rxtx.h"
42+
#include "en_accel/tls_rxtx.h"
43+
#include "en.h"
44+
45+
static inline struct sk_buff *mlx5e_accel_handle_tx(struct sk_buff *skb,
46+
struct mlx5e_txqsq *sq,
47+
struct net_device *dev,
48+
struct mlx5e_tx_wqe **wqe,
49+
u16 *pi)
50+
{
51+
#ifdef CONFIG_MLX5_EN_TLS
52+
if (sq->state & BIT(MLX5E_SQ_STATE_TLS)) {
53+
skb = mlx5e_tls_handle_tx_skb(dev, sq, skb, wqe, pi);
54+
if (unlikely(!skb))
55+
return NULL;
56+
}
57+
#endif
58+
59+
#ifdef CONFIG_MLX5_EN_IPSEC
60+
if (sq->state & BIT(MLX5E_SQ_STATE_IPSEC)) {
61+
skb = mlx5e_ipsec_handle_tx_skb(dev, *wqe, skb);
62+
if (unlikely(!skb))
63+
return NULL;
64+
}
65+
#endif
66+
67+
return skb;
68+
}
69+
70+
#endif /* CONFIG_MLX5_ACCEL */
71+
72+
#endif /* __MLX5E_EN_ACCEL_H__ */

‎drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec.h

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -45,9 +45,6 @@
4545
#define MLX5E_IPSEC_SADB_RX_BITS 10
4646
#define MLX5E_IPSEC_ESN_SCOPE_MID 0x80000000L
4747

48-
#define MLX5E_METADATA_ETHER_TYPE (0x8CE4)
49-
#define MLX5E_METADATA_ETHER_LEN 8
50-
5148
struct mlx5e_priv;
5249

5350
struct mlx5e_ipsec_sw_stats {

0 commit comments

Comments
 (0)