[libc++] Make _LIBCPP_EXTERN_TEMPLATE_TYPE_VIS export members

When building libc++ with hidden visibility, we want explicit template
instantiations to export members. This is consistent with existing
Windows behavior, and is necessary for clients to be able to link
against a hidden visibility built libc++ without running into lots of
missing symbols.

An unfortunate side effect, however, is that any template methods of a
class with an explicit instantiation will get default visibility when
instantiated, unless the methods are explicitly marked inline or hidden
visibility. This is not desirable for clients of libc++ headers who wish
to control their visibility, and led to PR30642.

Annotate all problematic methods with an explicit visibility specifier
to avoid this. The problematic methods were found by running
https://github.com/smeenai/bad-visibility-finder against the libc++
headers after making the _LIBCPP_EXTERN_TEMPLATE_TYPE_VIS change. The
methods were marked with the new _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
macro, which was created for this purpose.

It should be noted that _LIBCPP_EXTERN_TEMPLATE_TYPE_VIS was originally
intended to expand to default visibility, and was changed to expanding
to default type visibility to fix PR30642. The visibility macro
documentation was not updated accordingly, however, so this change makes
the macro consistent with its documentation again, while explicitly
fixing the methods which resulted in that PR.

Differential Revision: https://reviews.llvm.org/D29157

llvm-svn: 296731

Author: smeenai

libcxx/docs/DesignDocs/VisibilityMacros.rst

@@ -110,6 +110,35 @@ Visibility Macros
   the extern template declaration) as exported on Windows, as discussed above.
   On all other platforms, this macro has an empty definition.
 
+**_LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS**
+  Mark a symbol as hidden so it will not be exported from shared libraries. This
+  is intended specifically for method templates of either classes marked with
+  `_LIBCPP_TYPE_VIS` or classes with an extern template instantiation
+  declaration marked with `_LIBCPP_EXTERN_TEMPLATE_TYPE_VIS`.
+
+  When building libc++ with hidden visibility, we want explicit template
+  instantiations to export members, which is consistent with existing Windows
+  behavior. We also want classes annotated with `_LIBCPP_TYPE_VIS` to export
+  their members, which is again consistent with existing Windows behavior.
+  Both these changes are necessary for clients to be able to link against a
+  libc++ DSO built with hidden visibility without encountering missing symbols.
+
+  An unfortunate side effect, however, is that method templates of classes
+  either marked `_LIBCPP_TYPE_VIS` or with extern template instantiation
+  declarations marked with `_LIBCPP_EXTERN_TEMPLATE_TYPE_VIS` also get default
+  visibility when instantiated. These methods are often implicitly instantiated
+  inside other libraries which use the libc++ headers, and will therefore end up
+  being exported from those libraries, since those implicit instantiations will
+  receive default visibility. This is not acceptable for libraries that wish to
+  control their visibility, and led to PR30642.
+
+  Consequently, all such problematic method templates are explicitly marked
+  either hidden (via this macro) or inline, so that they don't leak into client
+  libraries. The problematic methods were found by running
+  `bad-visibility-finder <https://github.com/smeenai/bad-visibility-finder>`_
+  against the libc++ headers after making `_LIBCPP_TYPE_VIS` and
+  `_LIBCPP_EXTERN_TEMPLATE_TYPE_VIS` expand to default visibility.
+
 **_LIBCPP_EXTERN_TEMPLATE_INLINE_VISIBILITY**
   Mark a member function of a class template as visible and always inline. This
   macro should only be applied to member functions of class templates that are

libcxx/include/__config

@@ -591,6 +591,7 @@ namespace std {
 #define _LIBCPP_EXTERN_VIS          _LIBCPP_DLL_VIS
 #define _LIBCPP_EXCEPTION_ABI       _LIBCPP_DLL_VIS
 #define _LIBCPP_HIDDEN
+#define _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
 #define _LIBCPP_TEMPLATE_VIS
 #define _LIBCPP_FUNC_VIS_ONLY
 #define _LIBCPP_ENUM_VIS
@@ -614,6 +615,9 @@ namespace std {
 #endif
 #endif
 
+// The inline should be removed once PR32114 is resolved
+#define _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS inline _LIBCPP_HIDDEN
+
 #ifndef _LIBCPP_FUNC_VIS
 #if !defined(_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS)
 #define _LIBCPP_FUNC_VIS __attribute__ ((__visibility__("default")))
@@ -668,7 +672,7 @@ namespace std {
 
 #ifndef _LIBCPP_EXTERN_TEMPLATE_TYPE_VIS
 #  if !defined(_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS) && __has_attribute(__type_visibility__)
-#    define _LIBCPP_EXTERN_TEMPLATE_TYPE_VIS __attribute__ ((__type_visibility__("default")))
+#    define _LIBCPP_EXTERN_TEMPLATE_TYPE_VIS __attribute__ ((__visibility__("default")))
 #  else
 #    define _LIBCPP_EXTERN_TEMPLATE_TYPE_VIS
 #  endif

libcxx/include/locale

@@ -623,17 +623,20 @@ protected:
     ~num_get() {}
 
     template <class _Fp>
-    inline iter_type __do_get_floating_point
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    iter_type __do_get_floating_point
                             (iter_type __b, iter_type __e, ios_base& __iob,
                              ios_base::iostate& __err, _Fp& __v) const;
 
     template <class _Signed>
-    inline iter_type __do_get_signed
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    iter_type __do_get_signed
                             (iter_type __b, iter_type __e, ios_base& __iob,
                              ios_base::iostate& __err, _Signed& __v) const;
 
     template <class _Unsigned>
-    inline iter_type __do_get_unsigned
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    iter_type __do_get_unsigned
                             (iter_type __b, iter_type __e, ios_base& __iob,
                              ios_base::iostate& __err, _Unsigned& __v) const;
 

libcxx/include/string

@@ -792,6 +792,7 @@ public:
     basic_string(const basic_string& __str, size_type __pos,
                  const _Allocator& __a = _Allocator());
     template<class _Tp>
+        _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
         basic_string(const _Tp& __t, size_type __pos, size_type __n,
                      const allocator_type& __a = allocator_type(),
                      typename enable_if<__can_be_converted_to_string_view<_CharT, _Traits, _Tp>::value, void>::type* = 0);
@@ -927,7 +928,8 @@ public:
     basic_string& append(__self_view __sv) { return append(__sv.data(), __sv.size()); }
     basic_string& append(const basic_string& __str, size_type __pos, size_type __n=npos);
     template <class _Tp>
-    inline typename enable_if
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    typename enable_if
         <
             __can_be_converted_to_string_view<_CharT, _Traits, _Tp>::value,
             basic_string&
@@ -937,9 +939,11 @@ public:
     basic_string& append(const value_type* __s);
     basic_string& append(size_type __n, value_type __c);
     template <class _ForwardIterator>
-    inline basic_string& __append_forward_unsafe(_ForwardIterator, _ForwardIterator);
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    basic_string& __append_forward_unsafe(_ForwardIterator, _ForwardIterator);
     template<class _InputIterator>
-    inline typename enable_if
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    typename enable_if
         <
             __is_exactly_input_iterator<_InputIterator>::value
                 || !__libcpp_string_gets_noexcept_iterator<_InputIterator>::value,
@@ -952,7 +956,8 @@ public:
       return *this;
     }
     template<class _ForwardIterator>
-    inline typename enable_if
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    typename enable_if
         <
             __is_forward_iterator<_ForwardIterator>::value
                 && __libcpp_string_gets_noexcept_iterator<_ForwardIterator>::value,
@@ -988,7 +993,8 @@ public:
 #endif
     basic_string& assign(const basic_string& __str, size_type __pos, size_type __n=npos);
     template <class _Tp>
-    inline typename enable_if
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    typename enable_if
         <
             __can_be_converted_to_string_view<_CharT, _Traits, _Tp>::value,
             basic_string&
@@ -998,15 +1004,17 @@ public:
     basic_string& assign(const value_type* __s);
     basic_string& assign(size_type __n, value_type __c);
     template<class _InputIterator>
-    inline typename enable_if
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    typename enable_if
         <
            __is_exactly_input_iterator<_InputIterator>::value
                 || !__libcpp_string_gets_noexcept_iterator<_InputIterator>::value,
             basic_string&
         >::type
         assign(_InputIterator __first, _InputIterator __last);
     template<class _ForwardIterator>
-    inline typename enable_if
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    typename enable_if
         <
             __is_forward_iterator<_ForwardIterator>::value
                  && __libcpp_string_gets_noexcept_iterator<_ForwardIterator>::value,
@@ -1023,7 +1031,8 @@ public:
     _LIBCPP_INLINE_VISIBILITY
     basic_string& insert(size_type __pos1, __self_view __sv) { return insert(__pos1, __sv.data(), __sv.size()); }
     template <class _Tp>
-    inline typename enable_if
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    typename enable_if
         <
             __can_be_converted_to_string_view<_CharT, _Traits, _Tp>::value,
             basic_string&
@@ -1037,15 +1046,17 @@ public:
     _LIBCPP_INLINE_VISIBILITY
     iterator      insert(const_iterator __pos, size_type __n, value_type __c);
     template<class _InputIterator>
-    inline typename enable_if
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    typename enable_if
         <
            __is_exactly_input_iterator<_InputIterator>::value
                 || !__libcpp_string_gets_noexcept_iterator<_InputIterator>::value,
             iterator
         >::type
         insert(const_iterator __pos, _InputIterator __first, _InputIterator __last);
     template<class _ForwardIterator>
-    inline typename enable_if
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    typename enable_if
         <
             __is_forward_iterator<_ForwardIterator>::value
                  && __libcpp_string_gets_noexcept_iterator<_ForwardIterator>::value,
@@ -1070,7 +1081,8 @@ public:
     basic_string& replace(size_type __pos1, size_type __n1, __self_view __sv) { return replace(__pos1, __n1, __sv.data(), __sv.size()); }
     basic_string& replace(size_type __pos1, size_type __n1, const basic_string& __str, size_type __pos2, size_type __n2=npos);
     template <class _Tp>
-    inline typename enable_if
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    typename enable_if
         <
             __can_be_converted_to_string_view<_CharT, _Traits, _Tp>::value,
             basic_string&
@@ -1090,7 +1102,8 @@ public:
     _LIBCPP_INLINE_VISIBILITY
     basic_string& replace(const_iterator __i1, const_iterator __i2, size_type __n, value_type __c);
     template<class _InputIterator>
-    inline typename enable_if
+    _LIBCPP_METHOD_TEMPLATE_IMPLICIT_INSTANTIATION_VIS
+    typename enable_if
         <
             __is_input_iterator<_InputIterator>::value,
             basic_string&

libcxx/lib/abi/CHANGELOG.TXT

@@ -16,7 +16,7 @@ New entries should be added directly below the "Version" header.
 Version 5.0
 -----------
 
-* rTBD - Remove std::num_get template methods which should be inline
+* r296729 - Remove std::num_get template methods which should be inline
 
   These functions should never have had visible definitions in the dylib but
   since they were previously not specified with 'inline' they accidentally