Merge branch 'for-linus' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/jikos/livepatching

Pull livepatching updates from Jiri Kosina:

 - shadow variables support, allowing livepatches to associate new
   "shadow" fields to existing data structures, from Joe Lawrence

 - pre/post patch callbacks API, allowing livepatch writers to register
   callbacks to be called before and after patch application, from Joe
   Lawrence

* 'for-linus' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/jikos/livepatching:
  livepatch: __klp_disable_patch() should never be called for disabled patches
  livepatch: Correctly call klp_post_unpatch_callback() in error paths
  livepatch: add transition notices
  livepatch: move transition "complete" notice into klp_complete_transition()
  livepatch: add (un)patch callbacks
  livepatch: Small shadow variable documentation fixes
  livepatch: __klp_shadow_get_or_alloc() is local to shadow.c
  livepatch: introduce shadow variable API

Author: torvalds

Documentation/livepatch/callbacks.txt

@@ -0,0 +1,192 @@
+================
+Shadow Variables
+================
+
+Shadow variables are a simple way for livepatch modules to associate
+additional "shadow" data with existing data structures.  Shadow data is
+allocated separately from parent data structures, which are left
+unmodified.  The shadow variable API described in this document is used
+to allocate/add and remove/free shadow variables to/from their parents.
+
+The implementation introduces a global, in-kernel hashtable that
+associates pointers to parent objects and a numeric identifier of the
+shadow data.  The numeric identifier is a simple enumeration that may be
+used to describe shadow variable version, class or type, etc.  More
+specifically, the parent pointer serves as the hashtable key while the
+numeric id subsequently filters hashtable queries.  Multiple shadow
+variables may attach to the same parent object, but their numeric
+identifier distinguishes between them.
+
+
+1. Brief API summary
+====================
+
+(See the full API usage docbook notes in livepatch/shadow.c.)
+
+A hashtable references all shadow variables.  These references are
+stored and retrieved through a <obj, id> pair.
+
+* The klp_shadow variable data structure encapsulates both tracking
+meta-data and shadow-data:
+  - meta-data
+    - obj - pointer to parent object
+    - id - data identifier
+  - data[] - storage for shadow data
+
+It is important to note that the klp_shadow_alloc() and
+klp_shadow_get_or_alloc() calls, described below, store a *copy* of the
+data that the functions are provided.  Callers should provide whatever
+mutual exclusion is required of the shadow data.
+
+* klp_shadow_get() - retrieve a shadow variable data pointer
+  - search hashtable for <obj, id> pair
+
+* klp_shadow_alloc() - allocate and add a new shadow variable
+  - search hashtable for <obj, id> pair
+  - if exists
+    - WARN and return NULL
+  - if <obj, id> doesn't already exist
+    - allocate a new shadow variable
+    - copy data into the new shadow variable
+    - add <obj, id> to the global hashtable
+
+* klp_shadow_get_or_alloc() - get existing or alloc a new shadow variable
+  - search hashtable for <obj, id> pair
+  - if exists
+    - return existing shadow variable
+  - if <obj, id> doesn't already exist
+    - allocate a new shadow variable
+    - copy data into the new shadow variable
+    - add <obj, id> pair to the global hashtable
+
+* klp_shadow_free() - detach and free a <obj, id> shadow variable
+  - find and remove a <obj, id> reference from global hashtable
+    - if found, free shadow variable
+
+* klp_shadow_free_all() - detach and free all <*, id> shadow variables
+  - find and remove any <*, id> references from global hashtable
+    - if found, free shadow variable
+
+
+2. Use cases
+============
+
+(See the example shadow variable livepatch modules in samples/livepatch/
+for full working demonstrations.)
+
+For the following use-case examples, consider commit 1d147bfa6429
+("mac80211: fix AP powersave TX vs.  wakeup race"), which added a
+spinlock to net/mac80211/sta_info.h :: struct sta_info.  Each use-case
+example can be considered a stand-alone livepatch implementation of this
+fix.
+
+
+Matching parent's lifecycle
+---------------------------
+
+If parent data structures are frequently created and destroyed, it may
+be easiest to align their shadow variables lifetimes to the same
+allocation and release functions.  In this case, the parent data
+structure is typically allocated, initialized, then registered in some
+manner.  Shadow variable allocation and setup can then be considered
+part of the parent's initialization and should be completed before the
+parent "goes live" (ie, any shadow variable get-API requests are made
+for this <obj, id> pair.)
+
+For commit 1d147bfa6429, when a parent sta_info structure is allocated,
+allocate a shadow copy of the ps_lock pointer, then initialize it:
+
+#define PS_LOCK 1
+struct sta_info *sta_info_alloc(struct ieee80211_sub_if_data *sdata,
+				const u8 *addr, gfp_t gfp)
+{
+	struct sta_info *sta;
+	spinlock_t *ps_lock;
+
+	/* Parent structure is created */
+	sta = kzalloc(sizeof(*sta) + hw->sta_data_size, gfp);
+
+	/* Attach a corresponding shadow variable, then initialize it */
+	ps_lock = klp_shadow_alloc(sta, PS_LOCK, NULL, sizeof(*ps_lock), gfp);
+	if (!ps_lock)
+		goto shadow_fail;
+	spin_lock_init(ps_lock);
+	...
+
+When requiring a ps_lock, query the shadow variable API to retrieve one
+for a specific struct sta_info:
+
+void ieee80211_sta_ps_deliver_wakeup(struct sta_info *sta)
+{
+	spinlock_t *ps_lock;
+
+	/* sync with ieee80211_tx_h_unicast_ps_buf */
+	ps_lock = klp_shadow_get(sta, PS_LOCK);
+	if (ps_lock)
+		spin_lock(ps_lock);
+	...
+
+When the parent sta_info structure is freed, first free the shadow
+variable:
+
+void sta_info_free(struct ieee80211_local *local, struct sta_info *sta)
+{
+	klp_shadow_free(sta, PS_LOCK);
+	kfree(sta);
+	...
+
+
+In-flight parent objects
+------------------------
+
+Sometimes it may not be convenient or possible to allocate shadow
+variables alongside their parent objects.  Or a livepatch fix may
+require shadow varibles to only a subset of parent object instances.  In
+these cases, the klp_shadow_get_or_alloc() call can be used to attach
+shadow variables to parents already in-flight.
+
+For commit 1d147bfa6429, a good spot to allocate a shadow spinlock is
+inside ieee80211_sta_ps_deliver_wakeup():
+
+#define PS_LOCK 1
+void ieee80211_sta_ps_deliver_wakeup(struct sta_info *sta)
+{
+	DEFINE_SPINLOCK(ps_lock_fallback);
+	spinlock_t *ps_lock;
+
+	/* sync with ieee80211_tx_h_unicast_ps_buf */
+	ps_lock = klp_shadow_get_or_alloc(sta, PS_LOCK,
+			&ps_lock_fallback, sizeof(ps_lock_fallback),
+			GFP_ATOMIC);
+	if (ps_lock)
+		spin_lock(ps_lock);
+	...
+
+This usage will create a shadow variable, only if needed, otherwise it
+will use one that was already created for this <obj, id> pair.
+
+Like the previous use-case, the shadow spinlock needs to be cleaned up.
+A shadow variable can be freed just before its parent object is freed,
+or even when the shadow variable itself is no longer required.
+
+
+Other use-cases
+---------------
+
+Shadow variables can also be used as a flag indicating that a data
+structure was allocated by new, livepatched code.  In this case, it
+doesn't matter what data value the shadow variable holds, its existence
+suggests how to handle the parent object.
+
+
+3. References
+=============
+
+* https://github.com/dynup/kpatch
+The livepatch implementation is based on the kpatch version of shadow
+variables.
+
+* http://files.mkgnu.net/files/dynamos/doc/papers/dynamos_eurosys_07.pdf
+Dynamic and Adaptive Updates of Non-Quiescent Subsystems in Commodity
+Operating System Kernels (Kritis Makris, Kyung Dong Ryu 2007) presented
+a datatype update technique called "shadow data structures".

Documentation/livepatch/shadow-vars.txt

@@ -87,10 +87,35 @@ struct klp_func {
 	bool transition;
 };
 
+struct klp_object;
+
+/**
+ * struct klp_callbacks - pre/post live-(un)patch callback structure
+ * @pre_patch:		executed before code patching
+ * @post_patch:		executed after code patching
+ * @pre_unpatch:	executed before code unpatching
+ * @post_unpatch:	executed after code unpatching
+ * @post_unpatch_enabled:	flag indicating if post-unpatch callback
+ * 				should run
+ *
+ * All callbacks are optional.  Only the pre-patch callback, if provided,
+ * will be unconditionally executed.  If the parent klp_object fails to
+ * patch for any reason, including a non-zero error status returned from
+ * the pre-patch callback, no further callbacks will be executed.
+ */
+struct klp_callbacks {
+	int (*pre_patch)(struct klp_object *obj);
+	void (*post_patch)(struct klp_object *obj);
+	void (*pre_unpatch)(struct klp_object *obj);
+	void (*post_unpatch)(struct klp_object *obj);
+	bool post_unpatch_enabled;
+};
+
 /**
  * struct klp_object - kernel object structure for live patching
  * @name:	module name (or NULL for vmlinux)
  * @funcs:	function entries for functions to be patched in the object
+ * @callbacks:	functions to be executed pre/post (un)patching
  * @kobj:	kobject for sysfs resources
  * @mod:	kernel module associated with the patched object
  *		(NULL for vmlinux)
@@ -100,6 +125,7 @@ struct klp_object {
 	/* external */
 	const char *name;
 	struct klp_func *funcs;
+	struct klp_callbacks callbacks;
 
 	/* internal */
 	struct kobject kobj;
@@ -164,6 +190,14 @@ static inline bool klp_have_reliable_stack(void)
 	       IS_ENABLED(CONFIG_HAVE_RELIABLE_STACKTRACE);
 }
 
+void *klp_shadow_get(void *obj, unsigned long id);
+void *klp_shadow_alloc(void *obj, unsigned long id, void *data,
+		       size_t size, gfp_t gfp_flags);
+void *klp_shadow_get_or_alloc(void *obj, unsigned long id, void *data,
+			      size_t size, gfp_t gfp_flags);
+void klp_shadow_free(void *obj, unsigned long id);
+void klp_shadow_free_all(unsigned long id);
+
 #else /* !CONFIG_LIVEPATCH */
 
 static inline int klp_module_coming(struct module *mod) { return 0; }

include/linux/livepatch.h

@@ -1,3 +1,3 @@
 obj-$(CONFIG_LIVEPATCH) += livepatch.o
 
-livepatch-objs := core.o patch.o transition.o
+livepatch-objs := core.o patch.o shadow.o transition.o

kernel/livepatch/Makefile

@@ -54,11 +54,6 @@ static bool klp_is_module(struct klp_object *obj)
 	return obj->name;
 }
 
-static bool klp_is_object_loaded(struct klp_object *obj)
-{
-	return !obj->name || obj->mod;
-}
-
 /* sets obj->mod if object is not vmlinux and module is found */
 static void klp_find_object_module(struct klp_object *obj)
 {
@@ -285,6 +280,11 @@ static int klp_write_object_relocations(struct module *pmod,
 
 static int __klp_disable_patch(struct klp_patch *patch)
 {
+	struct klp_object *obj;
+
+	if (WARN_ON(!patch->enabled))
+		return -EINVAL;
+
 	if (klp_transition_patch)
 		return -EBUSY;
 
@@ -295,6 +295,10 @@ static int __klp_disable_patch(struct klp_patch *patch)
 
 	klp_init_transition(patch, KLP_UNPATCHED);
 
+	klp_for_each_object(patch, obj)
+		if (obj->patched)
+			klp_pre_unpatch_callback(obj);
+
 	/*
 	 * Enforce the order of the func->transition writes in
 	 * klp_init_transition() and the TIF_PATCH_PENDING writes in
@@ -388,13 +392,18 @@ static int __klp_enable_patch(struct klp_patch *patch)
 		if (!klp_is_object_loaded(obj))
 			continue;
 
-		ret = klp_patch_object(obj);
+		ret = klp_pre_patch_callback(obj);
 		if (ret) {
-			pr_warn("failed to enable patch '%s'\n",
-				patch->mod->name);
+			pr_warn("pre-patch callback failed for object '%s'\n",
+				klp_is_module(obj) ? obj->name : "vmlinux");
+			goto err;
+		}
 
-			klp_cancel_transition();
-			return ret;
+		ret = klp_patch_object(obj);
+		if (ret) {
+			pr_warn("failed to patch object '%s'\n",
+				klp_is_module(obj) ? obj->name : "vmlinux");
+			goto err;
 		}
 	}
 
@@ -403,6 +412,11 @@ static int __klp_enable_patch(struct klp_patch *patch)
 	patch->enabled = true;
 
 	return 0;
+err:
+	pr_warn("failed to enable patch '%s'\n", patch->mod->name);
+
+	klp_cancel_transition();
+	return ret;
 }
 
 /**
@@ -854,9 +868,15 @@ static void klp_cleanup_module_patches_limited(struct module *mod,
 			 * is in transition.
 			 */
 			if (patch->enabled || patch == klp_transition_patch) {
+
+				if (patch != klp_transition_patch)
+					klp_pre_unpatch_callback(obj);
+
 				pr_notice("reverting patch '%s' on unloading module '%s'\n",
 					  patch->mod->name, obj->mod->name);
 				klp_unpatch_object(obj);
+
+				klp_post_unpatch_callback(obj);
 			}
 
 			klp_free_object_loaded(obj);
@@ -906,13 +926,25 @@ int klp_module_coming(struct module *mod)
 			pr_notice("applying patch '%s' to loading module '%s'\n",
 				  patch->mod->name, obj->mod->name);
 
+			ret = klp_pre_patch_callback(obj);
+			if (ret) {
+				pr_warn("pre-patch callback failed for object '%s'\n",
+					obj->name);
+				goto err;
+			}
+
 			ret = klp_patch_object(obj);
 			if (ret) {
 				pr_warn("failed to apply patch '%s' to module '%s' (%d)\n",
 					patch->mod->name, obj->mod->name, ret);
+
+				klp_post_unpatch_callback(obj);
 				goto err;
 			}
 
+			if (patch != klp_transition_patch)
+				klp_post_patch_callback(obj);
+
 			break;
 		}
 	}