Thank you for clearly describing your questions and modeling challenges.
Question 1: Have I set "Line pairs" correctly? Does Line pairs correspond to "Resolution in pulses per revolution"?
The help menu for the coder defines the Line Pairs parameter as follows:
"Deﬁnes the amount of pairs of black and white lines and therefore the encoder resolution. For every line pair, 4 state transitions at A, B per revolution can be seen. The index pulse at I is set once per revolution when the angle crosses 0."
Generally, the definition agrees with the pulses per revolution from encoder vendors, and 1000 would be the appropriate value, but I would double check the vendor documentation to confirm.
Question 2: If I insert the PLECS block "Quadrature Encoder Counter" in the controller the following input signals top scheme...
Currently the two blocks are not IO compatible. There is an effort to resolve this in a forthcoming TSP release.
An interim workaround is to break the library link of the Incremental Encoder and change the offline implementation. You can do this by right clicking the component and selecting "Subsystem + Break link" and then "Subsystem + Unprotect". Then with a bit of copy/pasting you can wire the speed and angle inputs as output signals for the parent subsystem. The Quadrature Encoder Counter accepts angle and speed inputs in radians and rads/sec. You would need to add a bit more logic to have the direction and index pulse logic for the "s" output of the Quadrature Encoder Counter perform similar to the RT Box.
The code deployed to the RT Box would not be impacted by changes in the offline model.