Skip to content

Commit bcabbcc

Browse files
ohadbcohadbc
committed
rpmsg: add virtio-based remote processor messaging bus
Add a virtio-based inter-processor communication bus, which enables kernel drivers to communicate with entities, running on remote processors, over shared memory using a simple messaging protocol. Every pair of AMP processors share two vrings, which are used to send and receive the messages over shared memory. The header of every message sent on the rpmsg bus contains src and dst addresses, which make it possible to multiplex several rpmsg channels on the same vring. Every rpmsg channel is a device on this bus. When a channel is added, and an appropriate rpmsg driver is found and probed, it is also assigned a local rpmsg address, which is then bound to the driver's callback. When inbound messages carry the local address of a bound driver, its callback is invoked by the bus. This patch provides a kernel interface only; user space interfaces will be later exposed by kernel users of this rpmsg bus. Designed with Brian Swetland <[email protected]>. Signed-off-by: Ohad Ben-Cohen <[email protected]> Acked-by: Rusty Russell <[email protected]> (virtio_ids.h) Cc: Brian Swetland <[email protected]> Cc: Arnd Bergmann <[email protected]> Cc: Grant Likely <[email protected]> Cc: Tony Lindgren <[email protected]> Cc: Russell King <[email protected]> Cc: Andrew Morton <[email protected]> Cc: Greg KH <[email protected]> Cc: Stephen Boyd <[email protected]>
1 parent 34ed5a3 commit bcabbcc

File tree

10 files changed

+1739
-0
lines changed

10 files changed

+1739
-0
lines changed

‎Documentation/ABI/testing/sysfs-bus-rpmsg

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
What: /sys/bus/rpmsg/devices/.../name
2+
Date: June 2011
3+
KernelVersion: 3.3
4+
Contact: Ohad Ben-Cohen <[email protected]>
5+
Description:
6+
Every rpmsg device is a communication channel with a remote
7+
processor. Channels are identified with a (textual) name,
8+
which is maximum 32 bytes long (defined as RPMSG_NAME_SIZE in
9+
rpmsg.h).
10+
11+
This sysfs entry contains the name of this channel.
12+
13+
What: /sys/bus/rpmsg/devices/.../src
14+
Date: June 2011
15+
KernelVersion: 3.3
16+
Contact: Ohad Ben-Cohen <[email protected]>
17+
Description:
18+
Every rpmsg device is a communication channel with a remote
19+
processor. Channels have a local ("source") rpmsg address,
20+
and remote ("destination") rpmsg address. When an entity
21+
starts listening on one end of a channel, it assigns it with
22+
a unique rpmsg address (a 32 bits integer). This way when
23+
inbound messages arrive to this address, the rpmsg core
24+
dispatches them to the listening entity (a kernel driver).
25+
26+
This sysfs entry contains the src (local) rpmsg address
27+
of this channel. If it contains 0xffffffff, then an address
28+
wasn't assigned (can happen if no driver exists for this
29+
channel).
30+
31+
What: /sys/bus/rpmsg/devices/.../dst
32+
Date: June 2011
33+
KernelVersion: 3.3
34+
Contact: Ohad Ben-Cohen <[email protected]>
35+
Description:
36+
Every rpmsg device is a communication channel with a remote
37+
processor. Channels have a local ("source") rpmsg address,
38+
and remote ("destination") rpmsg address. When an entity
39+
starts listening on one end of a channel, it assigns it with
40+
a unique rpmsg address (a 32 bits integer). This way when
41+
inbound messages arrive to this address, the rpmsg core
42+
dispatches them to the listening entity.
43+
44+
This sysfs entry contains the dst (remote) rpmsg address
45+
of this channel. If it contains 0xffffffff, then an address
46+
wasn't assigned (can happen if the kernel driver that
47+
is attached to this channel is exposing a service to the
48+
remote processor. This make it a local rpmsg server,
49+
and it is listening for inbound messages that may be sent
50+
from any remote rpmsg client; it is not bound to a single
51+
remote entity).
52+
53+
What: /sys/bus/rpmsg/devices/.../announce
54+
Date: June 2011
55+
KernelVersion: 3.3
56+
Contact: Ohad Ben-Cohen <[email protected]>
57+
Description:
58+
Every rpmsg device is a communication channel with a remote
59+
processor. Channels are identified by a textual name (see
60+
/sys/bus/rpmsg/devices/.../name above) and have a local
61+
("source") rpmsg address, and remote ("destination") rpmsg
62+
address.
63+
64+
A channel is first created when an entity, whether local
65+
or remote, starts listening on it for messages (and is thus
66+
called an rpmsg server).
67+
68+
When that happens, a "name service" announcement is sent
69+
to the other processor, in order to let it know about the
70+
creation of the channel (this way remote clients know they
71+
can start sending messages).
72+
73+
This sysfs entry tells us whether the channel is a local
74+
server channel that is announced (values are either
75+
true or false).

‎Documentation/rpmsg.txt

Lines changed: 293 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,293 @@
1+
Remote Processor Messaging (rpmsg) Framework
2+
3+
Note: this document describes the rpmsg bus and how to write rpmsg drivers.
4+
To learn how to add rpmsg support for new platforms, check out remoteproc.txt
5+
(also a resident of Documentation/).
6+
7+
1. Introduction
8+
9+
Modern SoCs typically employ heterogeneous remote processor devices in
10+
asymmetric multiprocessing (AMP) configurations, which may be running
11+
different instances of operating system, whether it's Linux or any other
12+
flavor of real-time OS.
13+
14+
OMAP4, for example, has dual Cortex-A9, dual Cortex-M3 and a C64x+ DSP.
15+
Typically, the dual cortex-A9 is running Linux in a SMP configuration,
16+
and each of the other three cores (two M3 cores and a DSP) is running
17+
its own instance of RTOS in an AMP configuration.
18+
19+
Typically AMP remote processors employ dedicated DSP codecs and multimedia
20+
hardware accelerators, and therefore are often used to offload CPU-intensive
21+
multimedia tasks from the main application processor.
22+
23+
These remote processors could also be used to control latency-sensitive
24+
sensors, drive random hardware blocks, or just perform background tasks
25+
while the main CPU is idling.
26+
27+
Users of those remote processors can either be userland apps (e.g. multimedia
28+
frameworks talking with remote OMX components) or kernel drivers (controlling
29+
hardware accessible only by the remote processor, reserving kernel-controlled
30+
resources on behalf of the remote processor, etc..).
31+
32+
Rpmsg is a virtio-based messaging bus that allows kernel drivers to communicate
33+
with remote processors available on the system. In turn, drivers could then
34+
expose appropriate user space interfaces, if needed.
35+
36+
When writing a driver that exposes rpmsg communication to userland, please
37+
keep in mind that remote processors might have direct access to the
38+
system's physical memory and other sensitive hardware resources (e.g. on
39+
OMAP4, remote cores and hardware accelerators may have direct access to the
40+
physical memory, gpio banks, dma controllers, i2c bus, gptimers, mailbox
41+
devices, hwspinlocks, etc..). Moreover, those remote processors might be
42+
running RTOS where every task can access the entire memory/devices exposed
43+
to the processor. To minimize the risks of rogue (or buggy) userland code
44+
exploiting remote bugs, and by that taking over the system, it is often
45+
desired to limit userland to specific rpmsg channels (see definition below)
46+
it can send messages on, and if possible, minimize how much control
47+
it has over the content of the messages.
48+
49+
Every rpmsg device is a communication channel with a remote processor (thus
50+
rpmsg devices are called channels). Channels are identified by a textual name
51+
and have a local ("source") rpmsg address, and remote ("destination") rpmsg
52+
address.
53+
54+
When a driver starts listening on a channel, its rx callback is bound with
55+
a unique rpmsg local address (a 32-bit integer). This way when inbound messages
56+
arrive, the rpmsg core dispatches them to the appropriate driver according
57+
to their destination address (this is done by invoking the driver's rx handler
58+
with the payload of the inbound message).
59+
60+
61+
2. User API
62+
63+
int rpmsg_send(struct rpmsg_channel *rpdev, void *data, int len);
64+
- sends a message across to the remote processor on a given channel.
65+
The caller should specify the channel, the data it wants to send,
66+
and its length (in bytes). The message will be sent on the specified
67+
channel, i.e. its source and destination address fields will be
68+
set to the channel's src and dst addresses.
69+
70+
In case there are no TX buffers available, the function will block until
71+
one becomes available (i.e. until the remote processor consumes
72+
a tx buffer and puts it back on virtio's used descriptor ring),
73+
or a timeout of 15 seconds elapses. When the latter happens,
74+
-ERESTARTSYS is returned.
75+
The function can only be called from a process context (for now).
76+
Returns 0 on success and an appropriate error value on failure.
77+
78+
int rpmsg_sendto(struct rpmsg_channel *rpdev, void *data, int len, u32 dst);
79+
- sends a message across to the remote processor on a given channel,
80+
to a destination address provided by the caller.
81+
The caller should specify the channel, the data it wants to send,
82+
its length (in bytes), and an explicit destination address.
83+
The message will then be sent to the remote processor to which the
84+
channel belongs, using the channel's src address, and the user-provided
85+
dst address (thus the channel's dst address will be ignored).
86+
87+
In case there are no TX buffers available, the function will block until
88+
one becomes available (i.e. until the remote processor consumes
89+
a tx buffer and puts it back on virtio's used descriptor ring),
90+
or a timeout of 15 seconds elapses. When the latter happens,
91+
-ERESTARTSYS is returned.
92+
The function can only be called from a process context (for now).
93+
Returns 0 on success and an appropriate error value on failure.
94+
95+
int rpmsg_send_offchannel(struct rpmsg_channel *rpdev, u32 src, u32 dst,
96+
void *data, int len);
97+
- sends a message across to the remote processor, using the src and dst
98+
addresses provided by the user.
99+
The caller should specify the channel, the data it wants to send,
100+
its length (in bytes), and explicit source and destination addresses.
101+
The message will then be sent to the remote processor to which the
102+
channel belongs, but the channel's src and dst addresses will be
103+
ignored (and the user-provided addresses will be used instead).
104+
105+
In case there are no TX buffers available, the function will block until
106+
one becomes available (i.e. until the remote processor consumes
107+
a tx buffer and puts it back on virtio's used descriptor ring),
108+
or a timeout of 15 seconds elapses. When the latter happens,
109+
-ERESTARTSYS is returned.
110+
The function can only be called from a process context (for now).
111+
Returns 0 on success and an appropriate error value on failure.
112+
113+
int rpmsg_trysend(struct rpmsg_channel *rpdev, void *data, int len);
114+
- sends a message across to the remote processor on a given channel.
115+
The caller should specify the channel, the data it wants to send,
116+
and its length (in bytes). The message will be sent on the specified
117+
channel, i.e. its source and destination address fields will be
118+
set to the channel's src and dst addresses.
119+
120+
In case there are no TX buffers available, the function will immediately
121+
return -ENOMEM without waiting until one becomes available.
122+
The function can only be called from a process context (for now).
123+
Returns 0 on success and an appropriate error value on failure.
124+
125+
int rpmsg_trysendto(struct rpmsg_channel *rpdev, void *data, int len, u32 dst)
126+
- sends a message across to the remote processor on a given channel,
127+
to a destination address provided by the user.
128+
The user should specify the channel, the data it wants to send,
129+
its length (in bytes), and an explicit destination address.
130+
The message will then be sent to the remote processor to which the
131+
channel belongs, using the channel's src address, and the user-provided
132+
dst address (thus the channel's dst address will be ignored).
133+
134+
In case there are no TX buffers available, the function will immediately
135+
return -ENOMEM without waiting until one becomes available.
136+
The function can only be called from a process context (for now).
137+
Returns 0 on success and an appropriate error value on failure.
138+
139+
int rpmsg_trysend_offchannel(struct rpmsg_channel *rpdev, u32 src, u32 dst,
140+
void *data, int len);
141+
- sends a message across to the remote processor, using source and
142+
destination addresses provided by the user.
143+
The user should specify the channel, the data it wants to send,
144+
its length (in bytes), and explicit source and destination addresses.
145+
The message will then be sent to the remote processor to which the
146+
channel belongs, but the channel's src and dst addresses will be
147+
ignored (and the user-provided addresses will be used instead).
148+
149+
In case there are no TX buffers available, the function will immediately
150+
return -ENOMEM without waiting until one becomes available.
151+
The function can only be called from a process context (for now).
152+
Returns 0 on success and an appropriate error value on failure.
153+
154+
struct rpmsg_endpoint *rpmsg_create_ept(struct rpmsg_channel *rpdev,
155+
void (*cb)(struct rpmsg_channel *, void *, int, void *, u32),
156+
void *priv, u32 addr);
157+
- every rpmsg address in the system is bound to an rx callback (so when
158+
inbound messages arrive, they are dispatched by the rpmsg bus using the
159+
appropriate callback handler) by means of an rpmsg_endpoint struct.
160+
161+
This function allows drivers to create such an endpoint, and by that,
162+
bind a callback, and possibly some private data too, to an rpmsg address
163+
(either one that is known in advance, or one that will be dynamically
164+
assigned for them).
165+
166+
Simple rpmsg drivers need not call rpmsg_create_ept, because an endpoint
167+
is already created for them when they are probed by the rpmsg bus
168+
(using the rx callback they provide when they registered to the rpmsg bus).
169+
170+
So things should just work for simple drivers: they already have an
171+
endpoint, their rx callback is bound to their rpmsg address, and when
172+
relevant inbound messages arrive (i.e. messages which their dst address
173+
equals to the src address of their rpmsg channel), the driver's handler
174+
is invoked to process it.
175+
176+
That said, more complicated drivers might do need to allocate
177+
additional rpmsg addresses, and bind them to different rx callbacks.
178+
To accomplish that, those drivers need to call this function.
179+
Drivers should provide their channel (so the new endpoint would bind
180+
to the same remote processor their channel belongs to), an rx callback
181+
function, an optional private data (which is provided back when the
182+
rx callback is invoked), and an address they want to bind with the
183+
callback. If addr is RPMSG_ADDR_ANY, then rpmsg_create_ept will
184+
dynamically assign them an available rpmsg address (drivers should have
185+
a very good reason why not to always use RPMSG_ADDR_ANY here).
186+
187+
Returns a pointer to the endpoint on success, or NULL on error.
188+
189+
void rpmsg_destroy_ept(struct rpmsg_endpoint *ept);
190+
- destroys an existing rpmsg endpoint. user should provide a pointer
191+
to an rpmsg endpoint that was previously created with rpmsg_create_ept().
192+
193+
int register_rpmsg_driver(struct rpmsg_driver *rpdrv);
194+
- registers an rpmsg driver with the rpmsg bus. user should provide
195+
a pointer to an rpmsg_driver struct, which contains the driver's
196+
->probe() and ->remove() functions, an rx callback, and an id_table
197+
specifying the names of the channels this driver is interested to
198+
be probed with.
199+
200+
void unregister_rpmsg_driver(struct rpmsg_driver *rpdrv);
201+
- unregisters an rpmsg driver from the rpmsg bus. user should provide
202+
a pointer to a previously-registered rpmsg_driver struct.
203+
Returns 0 on success, and an appropriate error value on failure.
204+
205+
206+
3. Typical usage
207+
208+
The following is a simple rpmsg driver, that sends an "hello!" message
209+
on probe(), and whenever it receives an incoming message, it dumps its
210+
content to the console.
211+
212+
#include <linux/kernel.h>
213+
#include <linux/module.h>
214+
#include <linux/rpmsg.h>
215+
216+
static void rpmsg_sample_cb(struct rpmsg_channel *rpdev, void *data, int len,
217+
void *priv, u32 src)
218+
{
219+
print_hex_dump(KERN_INFO, "incoming message:", DUMP_PREFIX_NONE,
220+
16, 1, data, len, true);
221+
}
222+
223+
static int rpmsg_sample_probe(struct rpmsg_channel *rpdev)
224+
{
225+
int err;
226+
227+
dev_info(&rpdev->dev, "chnl: 0x%x -> 0x%x\n", rpdev->src, rpdev->dst);
228+
229+
/* send a message on our channel */
230+
err = rpmsg_send(rpdev, "hello!", 6);
231+
if (err) {
232+
pr_err("rpmsg_send failed: %d\n", err);
233+
return err;
234+
}
235+
236+
return 0;
237+
}
238+
239+
static void __devexit rpmsg_sample_remove(struct rpmsg_channel *rpdev)
240+
{
241+
dev_info(&rpdev->dev, "rpmsg sample client driver is removed\n");
242+
}
243+
244+
static struct rpmsg_device_id rpmsg_driver_sample_id_table[] = {
245+
{ .name = "rpmsg-client-sample" },
246+
{ },
247+
};
248+
MODULE_DEVICE_TABLE(rpmsg, rpmsg_driver_sample_id_table);
249+
250+
static struct rpmsg_driver rpmsg_sample_client = {
251+
.drv.name = KBUILD_MODNAME,
252+
.drv.owner = THIS_MODULE,
253+
.id_table = rpmsg_driver_sample_id_table,
254+
.probe = rpmsg_sample_probe,
255+
.callback = rpmsg_sample_cb,
256+
.remove = __devexit_p(rpmsg_sample_remove),
257+
};
258+
259+
static int __init init(void)
260+
{
261+
return register_rpmsg_driver(&rpmsg_sample_client);
262+
}
263+
module_init(init);
264+
265+
static void __exit fini(void)
266+
{
267+
unregister_rpmsg_driver(&rpmsg_sample_client);
268+
}
269+
module_exit(fini);
270+
271+
Note: a similar sample which can be built and loaded can be found
272+
in samples/rpmsg/.
273+
274+
4. Allocations of rpmsg channels:
275+
276+
At this point we only support dynamic allocations of rpmsg channels.
277+
278+
This is possible only with remote processors that have the VIRTIO_RPMSG_F_NS
279+
virtio device feature set. This feature bit means that the remote
280+
processor supports dynamic name service announcement messages.
281+
282+
When this feature is enabled, creation of rpmsg devices (i.e. channels)
283+
is completely dynamic: the remote processor announces the existence of a
284+
remote rpmsg service by sending a name service message (which contains
285+
the name and rpmsg addr of the remote service, see struct rpmsg_ns_msg).
286+
287+
This message is then handled by the rpmsg bus, which in turn dynamically
288+
creates and registers an rpmsg channel (which represents the remote service).
289+
If/when a relevant rpmsg driver is registered, it will be immediately probed
290+
by the bus, and can then start sending messages to the remote service.
291+
292+
The plan is also to add static creation of rpmsg channels via the virtio
293+
config space, but it's not implemented yet.

‎drivers/Kconfig

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,8 @@ source "drivers/iommu/Kconfig"
134134

135135
source "drivers/remoteproc/Kconfig"
136136

137+
source "drivers/rpmsg/Kconfig"
138+
137139
source "drivers/virt/Kconfig"
138140

139141
source "drivers/devfreq/Kconfig"

‎drivers/Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,7 @@ obj-$(CONFIG_HWSPINLOCK) += hwspinlock/
127127
obj-$(CONFIG_NFC) += nfc/
128128
obj-$(CONFIG_IOMMU_SUPPORT) += iommu/
129129
obj-$(CONFIG_REMOTEPROC) += remoteproc/
130+
obj-$(CONFIG_RPMSG) += rpmsg/
130131

131132
# Virtualization drivers
132133
obj-$(CONFIG_VIRT_DRIVERS) += virt/

‎drivers/rpmsg/Kconfig

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# RPMSG always gets selected by whoever wants it
2+
config RPMSG
3+
tristate
4+
select VIRTIO
5+
select VIRTIO_RING

‎drivers/rpmsg/Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
obj-$(CONFIG_RPMSG) += virtio_rpmsg_bus.o

0 commit comments

Comments
 (0)