wiki:Dev/Pointer

This page contains the discussions surrounding pointers.


Current Status

  • The current 2.x implementation can use either FCPtr or C style pointer.
  • C style pointer can be used with or without the multi-threading code enabled (not 100% commited GV).
  • <!> Compared to 1.x the ref counting inside OpenSG 2.x works correctly breaking code that relies on the old, broken implementation

Where to go

My goal would be to move the system to C-pointer only, and get rid of the whole FCPtr hierarchy and a lot of the associated template/trait magic (PointerBuilder etc.), as far as possible. I like Allen's idea of making the app-system interfaces return smart pointers, which could be boost::smart_ptr for a C-pointer style system, but don't have to be (I haven't looked at their implementation, but AFAIR they didn't do intrusive refcounts for compatibility reasons, which would be the most efficient). Does the system use RefPtrs internally now or is the ref counting done explicitly? We could use boost-pointers internally, too, but I'm not too hot for that. --DR

There is not that much magic ;-). Anyway some comments (GV) :

  • Using boost::smart_ptr for the API and store C-pointer internally is going to fail. The way smart_ptrs are build it is either use them

everywhere or not at all. Mixing them is not a good idea, mainly because every new smart_ptr will start its own refcounting ;-)

  • Refcounting is explict as we still have some loop possibilities (parent, beacon). But as it is part of the generated fcdProcess code

it should be quite equal to the use of RefPtrs with a little better control. I don't see a big benefit using RefPtr internally as

you have to use functions instead of direct field maniplation in order to make the other things (like change tacking, parent linking)

work while implementing your fieldcontainer. We could try to squeeze some of that into a new ptr class, but I'm not sure if this is

really a good thing to do or not.

  • As for Allens idea, basically it should be possible. I'm still struggling where to split. Something like getCore is internal as well

as app-system. So you might end up having two interfaces for the same thing (getting the core) without much chance of hiding one of

them from the user. But than there might not be that many places where you would have to duplicate the interface.

  • Iff the only problem with FCPtrs is that they do have a hierarchy and RefPtr or boost::smart_ptr don't another possibility would be

to get rid of it, which should not be that hard.

  • With C-pointers there is still the question where to do the aspect mapping. Explicit or within the pointer that travels the app-system

border (in case we decide for using something like a RefPtr class).

--GV

Comments marcusl:

  • As GV says, shared_ptr uses external ref-count. Not useful.
  • There is boost::intrusive_ptr which allows you to use an internal ref-count.
  • How about using weak-pointers to break cycles? (or, add gc since we can walk all fields of fc? :)
  • There is boost::dynamic_pointer_cast,etc for mapping between pointers to inherited classes.
  • I like the idea of do aspect mapping in refptr. Makes sense to use refptrs to communicate between threads anyway.

DR:

  • Mixing boost and non-boost: agreed.
  • Explicit refcounting: I would like to be able to use the reflective interface exclusively, as otherwise it doesn't make a lot of sense, IMHO. And for that to work all those updates need to happen in changed() anyway, so using RefPtrs might actually simplify some things. I'd hope to get around having a new, super-smart pointer. Maybe a weak ptr that is used for parent and beacon links and that is updated by the dependent changed()? This is easy for beacons, but the parent is not trivial as changed can't know which children are new, and even worse, which children have been removed and need updating. Maybe a weak ptr alleviates the need for updating removed children? It's not nice, as they still have a remnant link to their former parent, but a weak ptr would make it safe enough to avoid memory problems, even with that.
  • I don't think there are going to be many places with that problem. I would reallylike to make the code simpler to understand and less cary... ;)
  • Aspect mapping: I would go for the inter-thread pointer. I would keep the data in the class (to make it unique and accessible from the class, if necessary), but the actual mapping should be done in a special pointer class.

GV:

  • Pointer
  • weak_ptr work nicely with the smart_ptr due to the external blob that does the refcounting. weak_ptr force this blob to stay

around and the blob can be queried if the original pointer is still valid.

  • If we change to C-Pointer there would be a nice blob to misuse, the aspect store.
  • The biggest problem I see is that whatever we store (C Pointer or MagicPtr) the other side looses iterator access (well if we

don't want to build MagicIterators (TM) too ;-)).

  • This is kind of broken anyway. Without vbtl for the fields the whole generic interface becomes quite messy.
  • Currently there is a split in the generic interface, value fields can be manipulated directly (more or less) whereas

for pointer fields the 'parent' container provides generic access.

AB:

  • Pointer
  • I would not recommend trying to use boost::smart_ptr's for the fc ptr replacement. It is not designed for what we need. I think we could learn from their implementations and design something that solves our exact problems. As GV says, we need to have an internal refcount anyway and there are other constraints to take into account.
  • I would suggest using boost::shared_ptr for everything that is not an fc. So for example with ssm and the actions and things like that. All the objects that are not multi-buffered and are not put into the change list.
  • getCore split (gv): I had not thought about that one. I think you are right that we may have to create a few interfaces that are only used internally. I don't think we would want an addref/subref everytime an internal piece of code calls getCore. That could kill performance.
  • FCPtr problems (gv): My only problem with using fcptr's for fc's is that they still seem a little complex to understand (which is probably due more to the ability to use cptr's or fcptrs) and that they make users have to deal with ref counting manually. If we have smart_ptr fcptrs and internally used cptrs from those, I would be happy. but...
  • CPtrs (DR): Dirk, when you talk about using cptr's to hold everything is the main benefit performance? How would this code actually work. Where and how does aspect mapping happen and how will the various per-aspect fc's be allocated? Will each aspect have it's own block of memory and associated allocator so we can keep all the data for a single aspect "close" in memory? How expensive will this mapping be? Any code examples?
  • explicit refcounting (DR): I don't understand your comments here. Could you expand this use case into a full on example and put code above so we can all discuss the implications of how to implement some of these things?

GV:

  • CPtr
  • The trick is you have to map aspects a lot less than before. My par rendering app does exactly one explicit mapping for the window

and all the rest are C-Ptr references that stay within one aspect. I fixed that one recently so the code is currently not commited

completely.

  • Allocation currently uses new but there is nothing that prohibits custom allocators.
  • The expenses come during sync, for sfields instead of a simple assignment the pointer is mapped, for mfields you replace sharing magic

by a loop doing the mapping.

  • You loose a little bit memory as you can't share pointer fields anymore.
  • FCPtr problems, I see the point but MFields are still troublesome, because instead of a stored pointer you retrieve the

storage where the pointer is stored (Slight difference ;-)). So you have to support iterators and the [] operator in all variations.

The biggest problem is you can not overload on return types. Well you can fake it using templates but that would not

be to funny with [] ;-). Iterators might be ok because you can get away with a small wrapper and a different begin/end function.

Maybe scrapping the [] interface for pointer mfields might help, it's readonly anyway.

  • RefCounting
  • I think one of the problems we have right now is that it was broken in 1.x. As code is moved to 2.x, because suddenly ref counting

works, things break if the old code did not take care of correct ref counting.

  • So I'm still not sure if we have to pass out ref pointers. Because it should not make a difference if a ref pointer is passed

out and stored in a ref pointer outside or if a c-ptr is passed out and stored in a ref pointer. If this makes a difference

something else is badly broken.

  • I also think that the 2 1/2 rules covering refcounting are simple enough.
  • 'if you want to keep it call addRef'
  • 'create does not increment the ref count'
  • 'if you want to be lazy use refptrs'
Last modified 7 years ago Last modified on 01/17/10 01:11:44